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Preface 


The  research  presented  in  this  report  is  part  of  a  larger  project  that  examined  the 
effectiveness  of  several  notional  design  concepts  for  remotely  piloted  aircraft  (RPA).  Here,  we 
present  in  detail  the  analysis  of  a  pennissive  hunter-killer  mission. 

The  analysis  used  in  this  research  was  performed  using  the  RAND  Corporation’s  agent-based 
Systems  and  Concept  of  Operations  (CONOPS)  Operational  Effectiveness  Model  (SCOPEM), 
which  functions  within  the  Air  Force’s  agent-based  System  Effectiveness  Analysis  Simulation 
(SEAS)  modeling  environment.1  RAND  developed  additional  modeling  capabilities  for 
SCOPEM  during  fiscal  years  2009  and  2010  to  enable  this  work.2  This  report  also  serves  to 
document  these  capabilities  and  the  methodology  in  general. 

This  research  was  sponsored  by  Randall  Walden  (SAF/AQI),  Lt  Gen  James  (AF/A2),  and 
Maj  Gen  Andersen  (Air  Force  Intelligence,  Surveillance,  and  Reconnaissance  Agency).  The 
work  was  conducted  within  the  Force  Modernization  and  Employment  Program  of  RAND 
Project  AIR  FORCE  as  part  of  a  fiscal  year  2011  study,  “Developing  an  Analytically  Based 
Vision  for  Air  Force  Capabilities  Delivered  by  Remotely  Piloted  Aircraft.”  It  should  be  of 
interest  to  those  interested  in  the  trade-offs  between  different  RPA  design  concepts  and  to  those 
working  more  generally  in  the  fields  of  agent-based  modeling  and  simulation. 

RAND  Project  AIRFORCE 

RAND  Project  AIR  FORCE  (PAF),  a  division  of  the  RAND  Corporation,  is  the  U.S.  Air 
Force’s  federally  funded  research  and  development  center  for  studies  and  analyses.  PAF 
provides  the  Air  Force  with  independent  analyses  of  policy  alternatives  affecting  the 
development,  employment,  combat  readiness,  and  support  of  current  and  future  air,  space,  and 
cyber  forces.  Research  is  conducted  in  four  programs:  Force  Modernization  and  Employment; 
Manpower,  Personnel,  and  Training;  Resource  Management;  and  Strategy  and  Doctrine. 

Additional  information  about  PAF  is  available  on  our  website: 

http://www.rand.org/paf/ 


1  Lance  Menthe  and  Jeffrey  Sullivan,  A  RAND  Analysis  Tool  for  Intelligence,  Surveillance,  and  Reconnaissance: 
The  Collections  Operations  Model,  Santa  Monica,  Calif.:  RAND  Corporation,  TR-557-AF,  2008. 

2 

Sherrill  Lingel,  Lance  Menthe,  Brien  Alkire,  John  Gibson,  Scott  A.  Grossman,  Robert  A.  Guffey,  Keith  Henry, 
Lindsay  D.  Millard,  Christopher  A.  Mouton,  George  Nacouzi,  and  Edward  Wu,  Methodologies  for  Analyzing 
Remotely  Piloted  Aircraft  in  Future  Roles  and  Missions,  Santa  Monica,  Calif.:  RAND  Corporation,  DB-637-AF, 
2012. 
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Summary 


Introduction 

Remotely  piloted  aircraft  (RPAs) — notably  the  medium-altitude  MQ- 1  Predator  and  MQ-9 
Reaper — have  provided  crucial  support  to  counterinsurgency  and  counterterrorism  operations 
over  the  past  decade.  As  these  operations  come  to  a  close,  the  Air  Force  has  begun  to  evaluate 
additional  RPA  concepts  to  meet  future  needs. 

In  this  report,  we  analyze  the  operational  effectiveness  of  three  RPA  design  concepts,  plus 
the  MQ-9  as  a  baseline,  on  a  “hunter-killer”  mission:  to  find  and  destroy  a  specific  moving 
vehicle.  All  the  findings  in  this  report  are  based  on  the  modeling  results  for  a  particular  scenario 
(with  variants)  and  should  be  understood  within  that  context. 

Assumptions 

Although  our  scenario  is  representative  of  many  hunter-killer  missions  as  they  are  executed 
today,  it  is  not  representative  of  others.  It  also  relies  on  several  crucial  assumptions.  Most 
important,  the  threat  environment  we  modeled  is  permissive,  meaning  that  there  is  negligible 
threat  to  the  aircraft.  This  permits  the  use  of  concepts  of  operations  (CONOPS)  untempered  by 
survivability  concerns. 

In  the  particular  scenario  we  considered,  the  initial  cue  for  the  target  vehicle  is  urgent  but 
imprecise,  allowing  only  that  the  enemy  is  expected  to  emerge  from  hiding  somewhere  in  a  five 
nmi2  area  within  15  minutes.  Its  ultimate  destination  is  believed  to  be  outside  this  area  but  is 
otherwise  unknown.  Finding  the  target  vehicle  before  it  leaves  the  area  therefore  necessitates  a 
rapid,  brute-force  search. 

Because  the  area  is  not  especially  large,  the  search  is  not  difficult  but  is  complicated  by 
environmental  factors.  The  cued  area  is  an  urban  environment,  which  constrains  line  of  sight  to 
steep  viewing  angles.  Moreover,  in  many  of  the  variants  we  considered,  the  RPA  sensor  operator 
is  forced  to  contend  with  other  effects  detrimental  to  establishing  and  maintaining  line  of  sight, 
including  fog  and  clouds.  The  parameters  of  the  initial  cue,  as  described  above,  were  selected  to 
highlight  how  these  environmental  conditions  affect  operational  effectiveness. 

The  rules  of  engagement  and  CONOPS  we  selected  are  also  fairly  restrictive.  They  require 
that  the  target  be  positively  identified  with  full-motion  video  (FMV)  and  that  the  RPA  sensor 
operator  keep  the  target  in  view  with  FMV  whenever  weather  conditions  permit.  This  places  a 
high  burden  on  the  required  resolution  of  the  imagery,  which  must  yield  an  estimated  National 
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Imagery  Interpretability  Rating  Scale  (NIIRS)  quality  level  of  7.0  or  8.0. 1  (Both  cases  are 
modeled.)  While  reflective  of  current  practice,  these  requirements  effectively  relegate  ground 
moving  target  indication  (GMTI)  to  a  lesser  role  than  it  could  otherwise  play.  These 
requirements  also  preclude  the  RPA  from  flying  at  higher  altitudes,  which  in  turn  reduces  the 
effective  ground  coverage  area  that  the  RPA  sensor  operator  can  achieve.  The  rules  of 
engagement  therefore  force  the  RPA  pilot  to  choose  a  flight  profile  that  balances  the  competing 
demands  for  high  resolution  and  greater  ground  coverage  area. 

We  made  several  simplifying  operating  assumptions  as  well.  We  did  not  consider  the 
possibility  of  communications  jamming  or  loss  of  Global  Positioning  System  connectivity,  either 
of  which  could  potentially  be  devastating.  We  assumed  that  each  RPA  had  sufficient 
communications  capacity  and  processing,  exploitation,  and  dissemination  resources  to  fulfill  its 
mission — a  necessary  but  admittedly  heroic  assumption  mandated  largely  by  certain  technical 
limitations.  Finally,  while  the  enemy  is  allowed  some  awareness  that  it  is  in  danger  of  being 
watched,  we  assumed  that  it  could  neither  see  nor  hear  the  RPA.  The  target  vehicle  also  does  not 
employ  camouflage,  concealment,  or  any  of  the  other  clever  ruses  that  have  at  times  been 
observed  in  the  field:  The  only  adaptive  enemy  behavior  is  to  come  to  a  halt  from  time  to  time,  in 
an  attempt  to  confuse  possible  GMTI  tracking. 

These  choices  and  assumptions  enabled  us  to  avoid  certain  issues  so  that  we  could  focus 
instead  on  the  broader  trade-offs  between  platfonn  size  and  number;  sensor  performance;  and  the 
complicating  effects  of  darkness,  fog,  and  cloud  cover.  Flowever,  they  also  inherently  limit  the 
applicability  of  our  findings  regarding  these  trade-offs.  Nevertheless,  we  believe  that  our  analysis 
illuminates  some  important,  basic  considerations  in  RPA  design  choices.  It  also  demonstrates  a 
successful  analytical  methodology  that  may  be  expanded  to  address  more-challenging  scenarios. 

Design  Concepts 

The  RPA  design  concepts  we  considered  are  meant  to  represent  typical  parameters  for  what 
the  Air  Force  defines  today  as  “Group  3,”  “Group  4,”  and  “Group  5”  classes  of  RPAs,  with 
operational  performance  and  sensor  packages  comparable  to  those  of  today’s  aircraft  with  similar 
size,  weight,  and  power.3 4  These  designs  reflect  what  is  currently  practical  rather  than  what  may 


3 

For  a  description  of  the  NIIRS  scale,  see  Jon  Leachtenauer,  William  Malia,  John  Irvine,  Linda  Colburn,  and 
Nanette  Salvaggio,  “General  Image-Quality  Equation:  GIQE,”  Applied  Optics,  Vol.  36,  No.  32,  November  10,  1997. 

4 

These  groups  are  categorized  roughly  as  follows.  Group  3  is  anywhere  in  the  large  range  of  56-1,320  lbs.,  less 
than  250  knots  indicated  airspeed,  and  an  operating  altitude  below  18,000  ft.  above  mean  sea  level  (MSL).  Group  4 
is  anything  greater  than  1,320  lbs.,  at  any  airspeed,  with  the  same  restriction  on  operating  altitude  as  for  Group  3. 
Group  5  is  the  same  as  Group  4  but  with  an  operating  altitude  above  18,000  MSL  (U.S.  Department  of  Defense, 
Unmanned  Systems  Integrated  Roadmap  FY 2011-2036,  Washington,  D.C.,  November  9,  201  lb,  p.  12,  Figure  1). 

As  is  evident,  these  definitions  are  very  broad.  They  span  the  space  of  possibilities  but  only  because  they  specify  so 
little.  Future  research  on  more-advanced  RPA  concepts  would  benefit  from  a  richer  nomenclature. 
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be  possible  in  the  future.  Table  S.  1  outlines  the  configurations  of  these  designs,  with  the  MQ-9 
parameters  we  used  for  comparison. 

Modeling  Approach 

The  analysis  in  this  report  used  RAND’s  Systems  and  CONOPS  Operational  Effectiveness 
Model  (SCOPEM),  an  interlocking  suite  of  sensor,  environment,  and  platfonn  modules  built 
within  the  Air  Force’s  agent-based  System  Effectiveness  Analysis  Simulation  (SEAS)  modeling 
environment.5  SEAS  is  part  of  the  Air  Force  Standard  Analysis  Toolkit.  Using  this  model,  we 
performed  a  Monte  Carlo  analysis  of  each  of  the  144  variants  we  considered. 

Modeling  Results 

The  modeling  results  may  be  summarized  briefly  in  two  charts.  They  show  how  the 
operational  effectiveness  of  each  RPA  design  concept  varies  by  illumination  (day  versus  night), 
weather  (clear,  fog,  or  clouds),  number  of  platforms  (one  versus  three),6  and  using  different 
thresholds  for  positive  identification  of  the  target  vehicle  (NIIRS  7.0  versus  NIIRS  8.0). 

Figure  S.  1  below  shows  the  probability  that  the  target  vehicle  was  successfully  identified. 

Figure  S.2  shows  how  often  the  target  vehicle  was  also  successfully  followed  all  the  way  to  the 
kill  zone.  (Note  that  the  second  is  contingent  on  the  first.) 

Table  S.l.  RPA  Design  Concepts 


Concept 

Payload 

(lbs.) 

Cruise 

Speed 

(knots) 

Electro-Optical 
and  Infrared 
FMV  Sensor 

GMTI  and 
Synthetic 
Aperture  Radar 
Sensor 

MQ-9  Reaper 

3,850a 

180b 

MTS-B 

Lynx 

Group  5 

4,000 

360 

MTS-B 

Lynx 

Group  4 

535 

125 

Raven  Eye  II 

— 

Group  3 

35 

70 

Raven  Eye  1 

— 

a  Includes  850  lbs.  internal  and  3,000  lbs.  external  payload  capacity. 

bThe  actual  cruise  speed  varies.  We  selected  this  figure  for  consistency  of  analysis  with  other 
parts  of  the  study. 


5  Formerly  known  as  the  Collections  Operations  Model.  See  Lance  Menthe  and  Jeffrey  Sullivan,  A  RAND  Analysis 
Tool  for  Intelligence,  Surveillance,  and  Reconnaissance:  The  Collections  Operations  Model,  Santa  Monica,  Calif.: 
RAND  Corporation,  TR-557-AF,  2008. 

6  Intermediate  results  for  two  platforms  can  be  found  in  the  appendix. 
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Night  Day  Night 


Figure  S.l.  Summary  of  Probability  of  Identification 
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Figure  S.2.  Summary  of  Probability  of  Maintaining  Track  to  the  Kill  Zone 
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Conclusions 


From  the  analysis  that  underlies  these  summary  charts,  we  arrived  at  four  general  findings. 

First,  there  is  no  silver  bullet  for  RP A  performance  in  the  hunter-killer  mission.  Even  in  this 
highly  constrained  example,  no  single  RPA  concept  performed  well  on  all  measures  under  all 
environmental  conditions. 

Second,  numbers  can  compensate  for  capability.  In  this  scenario,  two  or  three  smaller  RPAs 
with  less-capable  sensor  packages  were  often  able  to  equal  or  exceed  the  performance  of  the 
larger  RPAs  employed  singly. 

Third,  the  MQ-9  holds  up  well  against  the  other  RPA  design  concepts  we  modeled  in  this 
scenario.  The  MQ-9  was  never  dramatically  outperformed  and  never  fared  worst  on  any 
measure.  It  compared  favorably  under  most  environmental  conditions. 

Finally,  improving  MQ-9  sensor  capabilities  may  be  a  cost-effective  option.  Although  we  did 
not  perform  a  cost-effectiveness  analysis,  upgrading  the  sensors  on  existing  MQ-9  platforms,  in 
particular  their  FMV  sensor  capabilities,  would  permit  greater  operational  flexibility  and  would 
offer  enhanced  operational  effectiveness  for  this  type  of  hunter-killer  scenario.  Furthermore,  if 
the  discrete  levels  of  magnification  currently  available  on  the  Multispectral  Targeting  System, 
Model  B  (MTS-B)  could  be  replaced  with  a  continuous  zoom  feature,  MTS-B  could  be  used 
more  effectively  to  enable  the  RPA  pilot  and  sensor  operator  to  balance  competing  mission 
objectives. 
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1.  Introduction 


Background 

Air  Force  remotely  piloted  aircraft  (RPAs) — in  particular,  the  medium-altitude,  multirole 
platforms,  the  MQ-1  Predator  and  MQ-9  Reaper — have  provided  crucial  support  to 
counterinsurgency  and  counterterrorism  operations  over  the  past  decade.7  Flowever,  the  last  of 
the  Predators  has  now  rolled  off  the  assembly  line,8  and  Air  Force  orders  for  new  Reapers  have 
reached  a  steady  state.9  To  meet  future  needs,  the  Air  Force  has  now  begun  to  assess  concepts  for 
a  potential  follow-on  RPA  system.10 

The  research  presented  in  this  report  is  part  of  a  larger  study  that  examined  the  effectiveness 
of  several  notional  design  concepts  for  RPAs.  Here,  we  present  in  detail  the  analysis  of  a 
pennissive  hunter-killer  mission. 

In  this  report,  we  analyze  the  operational  effectiveness  of  three  RPA  design  concepts,  plus 
the  MQ-9  as  a  baseline,  on  a  hunter-killer  mission:  to  find  and  destroy  a  specific  moving  vehicle. 
All  the  findings  in  this  report  are  based  on  the  modeling  results  for  a  particular  scenario  (with 
variants)  and  should  be  understood  within  that  context. 

Study  Approach 

Figure  1 . 1  outlines  the  overall  analytical  approach.  The  heart  of  the  analysis  was  performed 
using  RAND’s  Systems  and  Concepts  of  Operations  (CONOPS)  Operational  Effectiveness 
Model  (SCOPEM),11  an  interlocking  suite  of  sensor,  environment,  and  platform  modules  built 


7 

The  nomenclature  for  these  aircraft  can  be  somewhat  vexing,  because  the  MQ-1  Predator  has  many  offspring.  The 
MQ-9  Reaper  was  originally  conceived  as  an  improved  Predator  and  is  still  sometimes  referred  to  as  the 
“Predator  B” — which  is  not  to  be  confused  with  the  MQ-1B,  the  designation  for  a  newer,  upgraded  version  of  the 
original  Predator.  Similarly,  General  Atomics  refers  to  its  next-generation  RPA  system  as  the  “Predator  C”  Avenger, 
which  is  not  to  be  confused  with  the  Army’s  MQ-1C  Grey  Eagle  (formerly  Sky  Warrior),  an  even  newer  variant  of 
the  Predator.  Finally,  it  is  worth  noting  that  the  MQ-1  was  formerly  designated  the  RQ-1  prior  to  the  addition  of 
weapons  in  2001.  This  older  designation  is  still  used  on  occasion  to  emphasize  that  a  particular  airframe  carries  no 
weapons. 

g 

88th  Air  Base  Wing  Public  Affairs,  “Air  Force  Accepts  Delivery  of  Final  MQ-1  Predator,”  Wright-Patterson  Air 
Force  Base,  Ohio,  March  4,  2011. 

9 

Recent  planning  calls  for  a  fixed  quantity  of  48  aircraft  per  year  for  the  next  several  years.  See  U.S.  Department  of 
Defense  (DoD),  Aircraft  Procurement  Plan  Fiscal  Years  2012-2041,  Washington,  D.C.,  A-A78844C,  March  201  la, 
Chart  8. 

10  Dave  Majumdar,  “Air  Force  Orders  Single  Predator  C  Avenger,”  Air  Force  Times,  December  13,  2011. 

1 1  Formerly  known  as  the  Collections  Operations  Model.  See  Lance  Menthe  and  Jeffrey  Sullivan,  A  RAND  Analysis 
Tool  for  Intelligence,  Surveillance,  and  Reconnaissance:  The  Collections  Operations  Model,  Santa  Monica,  Calif.: 
RAND  Corporation,  TR-557-AF,  2008. 
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Figure  1.1.  Diagram  of  Methodology 


within  the  Air  Force’s  agent-based  System  Effectiveness  Analysis  Simulation  (SEAS)  modeling 
environment.  SEAS  is  part  of  the  Air  Force  Standard  Analysis  Toolkit.  Using  this  model,  we 
performed  a  Monte  Carlo  analysis  of  each  of  the  144  variants  of  the  hunter-killer  scenario  we 
considered. 


Design  Concepts  Considered 

The  RPA  design  concepts  we  examined  were  developed  as  part  of  the  larger  study.  The  study 
overview  document  discusses  these  concepts  and  their  development  in  more  detail. 

The  concepts  are  meant  to  represent  typical  parameters  for  what  the  Air  Force  defines  today 
as  “Group  3,”  “Group  4,”  and  “Group  5”  classes  of  RPAs,  with  operational  performance  and 
sensor  packages  comparable  to  those  of  today’s  aircraft  with  similar  size,  weight,  and  power.12 
The  designs  are  not  radical  but  were  selected  in  an  attempt  to  analyze  middle-of-the-road  options 
with  conservative  assumptions  about  future  capabilities.  Thus,  the  RPA  design  concepts  we 
assessed  reflect  what  is  currently  practical  rather  than  what  may  be  possible  in  the  future. 

Table  1.1  summarizes  the  basic  specifications  of  the  design  concepts  and  provides 
comparable  data  for  the  MQ-9. 


12 

These  groups  are  categorized  roughly  as  follows.  Group  3  is  anywhere  in  the  large  range  of  56-1320  lbs.,  less 
than  250  knots  indicated  airspeed,  and  an  operating  altitude  below  18,000  ft.  above  mean  sea  level  (MSL).  Group  4 
is  anything  greater  than  1320  lbs.  at  any  airspeed,  with  the  same  restriction  on  operating  altitude  as  for  Group  3. 
Group  5  is  the  same  as  Group  4  but  with  an  operating  altitude  above  18,000  MSL  (DoD,  Unmanned  Systems 
Integrated  Roadmap  FY  2011-2036,  Washington,  D.C.,  November  9,  201  lb,  p.  12,  Figure  1).  As  is  evident,  these 
definitions  are  very  broad.  They  span  the  space  of  possibilities,  but  only  because  they  specify  so  little.  Future 
research  on  more  advanced  RPA  concepts  would  benefit  from  a  richer  nomenclature. 
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Table  1.1.  RPA  Design  Concepts 


Concept 

Payload 

(lbs.) 

Cruise 

Speed 

(knots) 

Electro-Optical 
(EO)  and 
Infrared  (IR) 
Full-Motion 
Video  (FMV) 
Sensor 

GMTI  and 
Synthetic 
Aperture  Radar 
(SAR) 

Sensor 

MQ-9  Reaper 

3,850a 

180t> 

MTS-B 

Lynx 

Group  5 

4,000 

360 

MTS-B 

Lynx 

Group  4 

535 

125 

Raven  Eye  II 

— 

Group  3C 

35 

70 

Raven  Eye  1 

— 

a  Includes  850  lbs.  internal  and  3,000  lbs.  external  payload  capacity. 
b  The  actual  cruise  speed  varies.  We  selected  this  figure  for  consistency  of  analysis. 
c  This  is  the  “light”  version.  A  “heavy”  version  was  included  in  other  parts  of  the  study,  but  was 
omitted  from  this  analysis  due  to  the  similarity  in  performance  and  capabilities  in  this  scenario. 


We  chose  not  to  examine  the  smallest  RPA  concepts,  i.e.,  Group  1  (micro-  and  mini-tactical) 
and  Group  2  (small  tactical)  for  this  particular  hunter-killer  scenario.  Despite  their  potential 
utility  for  many  other  missions,  these  smaller  RPAs  can  carry  only  very  light  payloads  and  must 
fly  at  low  altitudes,  which  would  require  a  very  different  suite  of  sensors  and  CONOPS  to  be 
successful  in  this  scenario.  Also,  as  a  practical  matter,  these  concepts  were  not  developed  as  part 
of  the  larger  study.  The  employment  possibilities  and  operational  effectiveness  of  smaller  RPAs 
in  missions  like  this  one  merit  further  research. 

Organization  of  This  Report 

Section  2  describes  the  modeling  approach  and  the  sensor  and  environmental  modules  within 
SCOPEM  that  are  relevant  to  the  analysis.  Section  3  details  the  particular  hunter-killer  scenario 
we  modeled.  Section  4  presents  the  modeling  results,  and  Section  5  summarizes  the  findings  that 
were  drawn  from  these  results.  Finally,  the  appendix  contains  a  condensed  version  of  the 
complete  numerical  modeling  output. 
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2.  Modeling  Approach 


Modeling  Environment 

The  underlying  modeling  environment,  SEAS,  is  a  stochastic,  programmable,  multiagent 
simulation  sponsored  by  the  Air  Force  Space  Command,  Space  and  Missile  Systems  Center, 
Directorate  of  Development  and  Planning.13  SEAS  is  part  of  the  Air  Force  Standard  Analysis 
Toolkit  and  the  Air  Force  Space  Command  Modeling  and  Simulation  Toolkit.  SEAS  has  been  in 
use  in  one  form  or  another  for  over  fifteen  years.  The  latest  stable  release  (March  2011)  is 
version  3.9. 14  SEAS  is  written  in  C++  and  runs  on  the  Microsoft  Windows  platform. 

SCOPEM  is  RAND’s  unique  modular  approach  to  scenario  construction  in  SEAS,  designed 
to  add  richness  and  to  facilitate  exploratory  modeling  of  variations  at  the  mission  level. 

Figure  2.1  depicts  the  conceptual  relationship  between  SCOPEM  and  SEAS  to  the  extent  that 
SCOPEM  is  a  suite  of  models  within  the  SEAS  modeling  environment.  Note  that,  while  SEAS 
provides  the  scaffolding,  all  the  systems  and  their  interactions  belong  to  SCOPEM.15 

An  agent  in  SEAS  can  be  programmed  to  do  almost  anything;  however,  the  agent  requires 
programming  for  almost  everything  it  does.16  The  situation  is  analogous  to  working  with 
Microsoft  Excel,  in  that  the  environment  can  be  used  to  do  many  calculations,  but  it  loads  as  an 
empty  sheet.  Rather  than  generating  specialized,  one-off  models  for  each  scenario — the  standard 
approach  to  SEAS  model  development — SCOPEM  builds  all  scenarios  from  common  pieces. 
Today,  SCOPEM’s  library  encompasses  models  of  hundreds  of  sensors,  weapons,  and  platforms. 
Figure  2.2  offers  an  example  of  the  graphical  output  of  SEAS. 


13 

First  created  by  the  Aerospace  Corporation,  SEAS  is  now  maintained  and  developed  for  the  Air  Force  by 
ExoAnalytic  Solutions. 

14 

Because  the  latest  version  was  released  after  this  study  was  already  well  under  way,  however,  the  authors  decided 
to  continue  work  with  the  previous  version  rather  than  change  horses  in  midstream.  SEAS  version  3.8hl  was  used 
for  this  research. 

15  Although  SEAS  itself  is  written  in  C++,  SCOPEM  is  written  in  the  Tactical  Programming  Language,  a  custom 
language  used  to  script  agent  behaviors  within  SEAS. 

16  What  exactly  constitutes  a  software  “agent”  is  a  controversy  we  do  not  engage  here.  For  our  purposes,  we  speak 
of  an  agent  as  that  entity  within  SEAS  that  has  its  own  perceptions  and  can  be  programmed  to  make  decisions  based 
on  those  perceptions. 
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Figure  2.1.  SCOPEM  is  a  Library  of  Interacting  Models  in  the 
SEAS  Modeling  Environment 
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Figure  2.2.  Example  of  the  Graphical  Output  of  SEAS 
Running  SCOPEM 
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As  a  mission-level  model,  SCOPEM  represents  each  platfonn,  sensor,  and  weapon  separately 
but  does  not  simulate  the  finer  details  of  these  systems — e.g.,  aerodynamics  and  waveform 
propagation  are  not  simulated  explicitly.  Instead,  more-detailed  models  of  these  other 
phenomena  are  used  to  generate  the  necessary  representations  of  sensor  performance  and 
environmental  conditions  in  SCOPEM.  These  other  models  are  documented  elsewhere.17  The 
interactions  between  these  models,  however,  are  within  SCOPEM  itself.  Although  these 
interactions  can  become  quite  complex,  the  core  behavioral  loop  common  to  nearly  all  agents  is 
the  dynamic  interaction  between  the  sensor  and  target,  as  mediated  by  the  environment. 

Figure  2.3  depicts  this  loop. 


Figure  2.3.  Overview  of  SCOPEM’s  Primary  Loop 


17 

Sherrill  Lingel,  Lance  Menthe,  Brien  Alkire,  John  Gibson,  Scott  A.  Grossman,  Robert  A.  Guffey,  Keith  Henry, 
Lindsay  D.  Millard,  Christopher  A.  Mouton,  George  Nacouzi,  and  Edward  Wu,  Methodologies  for  Analyzing 
Remotely  Piloted  Aircraft  in  Future  Roles  and  Missions,  Santa  Monica,  Calif.:  RAND  Corporation,  DB-637-AF, 


2012. 
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In  the  past,  SCOPEM  has  been  used  to  model  a  range  of  different  operations,  from  maritime 
surveillance  to  nontraditional  intelligence,  surveillance,  and  reconnaissance,  and  different 
modules  have  been  added  with  each  study.  In  the  subsequent  sections,  we  describe  the  specific 
sensor  and  environment  modules  developed  and  used  for  the  hunter-killer  mission. 

Although  SCOPEM  also  contains  weapon  models,  we  chose  not  to  model  the  entire  “kill” 
portion  of  the  scenario  within  this  analytic  framework.  Some  of  the  airframes  considered  are  too 
small  to  carry  both  weapons  and  sensors,  and  CONOPS  for  calling  in  other  aircraft  would  have 
been  required  for  a  fair  comparison.  Instead,  we  made  the  critical  assumption  that,  if  the 
remaining  stringent  conditions  for  a  successful  hunter-killer  mission  were  met — if  the  target 
could  be  found,  identified,  tracked  to  a  designated  kill  zone,  and  maintained  continuously  in 
track  for  a  sufficiently  long  time — the  RPA  could  either  kill  the  target  itself  or,  depending  on  the 
circumstances,  call  in  another  aircraft  to  do  so.  We  did  not  model  varying  weapon  performance. 

Sensor  Models 

Three  particular  sensor  modalities  played  a  role  in  this  study:  motion  imagery,  specifically 
FMV;  GMTI  with  an  associated  tracking  system;  and  SAR.  Each  of  these  modalities  has  a 
different  representation  within  SCOPEM.  Here,  we  describe  the  key  assumptions  involved,  those 
important  to  understanding  and  qualifying  the  results  of  the  analysis  of  the  hunter-killer  mission. 
In  all  these  models,  we  made  the  heroic  assumption  that  the  RPA  had  sufficient  communications 
and  processing,  exploitation,  and  dissemination  (PED)  capacity.  We  did  so  to  focus  on  sensor 
performance  characteristics.  In  reality,  PED  constraints  can  be  quite  significant,  and  the  issue 
merits  further  attention.  In  keeping  with  the  assumption  of  an  effectively  permissive 
environment,  we  also  did  not  consider  such  effects  as  jamming  or  the  loss  of  Global  Positioning 
System  connectivity. 

Full-Motion  Video 

Because  FMV  is  the  only  sensor  modality  employed  in  this  scenario  capable  of  making  a 
clear,  positive  identification  of  the  target,  the  way  in  which  FMV  image  quality  is  modeled  can 
strongly  affect  the  scenario  outcome.  Although  alternatives  to  FMV  certainly  exist  for  this 
purpose,  we  chose  to  emphasize  FMV  because  of  the  way  it  is  treated  in  similar  missions  today. 
Note  that  the  standard  specifications  for  FMV  sensors — resolution,  focal  length,  etc. — are  critical 
but  are  not  in  themselves  sufficient  to  determine  image  quality,  although  they  are  certainly  good 
indicators.18  In  SCOPEM  we  use  a  modified  version  of  the  general  image  quality  equation 


1 8 

Where  possible,  of  course,  we  would  always  prefer  to  use  actual  performance  data  to  characterize  sensor 
capabilities,  such  as  image  quality;  however,  this  was  not  (and  rarely  is)  possible. 
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(GIQE)  to  estimate  image  quality  based  on  these  standard  sensor  specifications  and  sensor-target 
geometry.19 

The  GIQE  is  a  regression-based  predictor  of  image  quality  in  terms  of  the  National  Imagery 
Interpretability  Rating  Scale  (NIIRS),  an  empirical  zero-to-ten  scale  that  describes  the  types  of 
exploitation  tasks  an  image  may  be  expected  to  support.20  Working  with  trained  imagery 
analysts,  U.S.  government  agencies  have  developed  tables  that  estimate  the  NIIRS  levels 
required  to  detect,  classify,  or  identify  various  objects  within  an  image;  we  use  these  tables 
together  with  the  GIQE  to  decide  whether  an  image  is  of  sufficient  quality  to  identify  the 
target.21  The  GIQE  is  an  unbiased  estimator  of  NIIRS,  with  a  standard  deviation  of  0.3  levels.22 

SCOPEM  uses  a  simplified  version  of  the  GIQE  for  its  calculations.  The  dominant  parameter 
in  the  GIQE  is  the  ground  sample  distance  (GSD),  a  combination  of  angular  resolution  and 
sensor-target  geometry  (see  Figure  2.4).  The  simplified  equation  implemented  in  SCOPEM 
replaces  most  other  parameters  (e.g.,  the  relative  edge  response  of  various  surfaces,  which  is  very 
difficult  to  predict)  with  their  average  values,  as  determined  from  the  initial  sampling  of  images 
used  to  create  the  GIQE.  To  the  extent  that  these  parameters  are  not  correlated  with  other  aspects 
of  the  scenario,  this  simplification  should  not  be  expected  to  make  a  significant  difference 
because  the  Monte  Carlo  nature  of  the  simulation  would  ultimately  average  over  these 
parameters  anyway.  However  certain  effects  of  the  environment  on  image  quality,  such  as  line- 
of-sight  (LOS)  obstructions  and  atmospheric  effects,  are  likely  to  be  correlated.  We  model  the 
impact  of  these  environmental  effects  separately,  as  discussed  in  the  next  subsection. 

Camouflage  and  concealment  would  also  affect  the  ability  to  identify  the  target  in  an  image. 
Because  these  cannot  be  treated  effectively  in  a  simple  manner,  our  simulation  is  limited  to  the 
cases  in  which  the  target  does  not  employ  these  techniques.  Our  assumptions  are  therefore 
optimistic  in  this  regard. 

All  the  FMV  sensors  modeled  in  this  study  are  in  fact  suites  of  sensors  with  both  EO  (visible) 
and  IR  components.  EO  NIIRS  and  IR  NIIRS  are  separate  scales,  and  there  are  somewhat 
different  versions  of  the  GIQE  for  each.  The  EO  and  IR  NIIRS  scales  are  not  technically 
identical  in  composition,  meaning  that  an  EO  NIIRS  of  a  certain  level  does  not  promise  the  same 
ability  to  detect,  classify,  and  identify  objects  as  an  IR  NIIRS  of  the  same  level;  these  differences 
are  accounted  for  in  the  modeling. 


19 

Jon  C.  Leachtenauer,  William  Malia,  John  Irvine,  Linda  Colburn,  and  Nanette  Salvaggio,  “General  Image-Quality 
Equation:  GIQE"  Applied  Optics,  Vol.  36,  No.  32,  November  10,  1997. 

20 

John  M.  Irvine,  Ana  Ivelisse  Aviles,  David  M.  Cannon,  Charles  Fenimore,  Donna  S.  Haverkamp,  Steven  A. 

Israel,  Gary  O’Brien,  and  John  Roberts,  “Developing  an  Interpretability  Scale  for  Motion  Imagery,”  Optical 
Engineering,  Vol.  46,  No.  11,  p.  117401,  November  2007. 

21 

Imagery  Resolution  Assessments  and  Reporting  Standards  Committee,  Civil  NIIRS  Reference,  March  1996. 

22 

As  NIIRS  is  an  integer  scale,  we  treat  the  integers  as  thresholds.  We  explicitly  model  the  0.3  NIIRS  variation 
within  SCOPEM. 
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Figure  2.4.  Ground  Sample  Distance 


NOTE:  The  actual  GSD  used  in  the  GIQE  (and  SCOPEM)  is  the  geometric  mean  of  the  cross-range  (GSDy)  and 
downrange  (GSDx)  terms.  If  the  image  is  skewed  such  that  these  directions  are  not  orthogonal,  the  GSD  must  be 
further  modified. 

Figure  2.5  illuminates  one  curiosity  of  the  GIQE  worth  noting.  All  other  things  being  equal, 
the  EO  and  IR  versions  of  the  GIQE  would  estimate  the  interpretability  of  an  IR  image  to  be 
better  than  that  of  the  corresponding  EO  image  with  the  same  GSD.  This  apparent  contradiction 
with  experience  is  resolved  by  observing  that  all  other  things  are  usually  not  equal.  In  particular, 
the  resolution  of  the  IR  sensor  is  usually  poorer  than  that  of  the  corresponding  EO  sensor  in  the 
same  mode,  which  leads  to  a  coarser  GSD. 

However,  this  does  highlight  a  particular  concern  with  the  version  of  the  GIQE  implemented 
in  SCOPEM.  As  mentioned  above,  we  invoke  a  simplified  version  in  which  the  values  of  most 
other  parameters  are  replaced  by  their  averages.  This  practice  is  more  problematic  for  IR  images, 
however,  because  some  of  the  GIQE  parameters,  notably  the  contrast,  vary  considerably  between 
night  and  day  and  on  the  ambient  temperature  of  the  background  clutter.  They  may  even  vary 
across  the  image  itself,  making  some  features  more  difficult  to  discern  than  others.  For  this 
reason,  the  IR  results  reported  here  should  be  viewed  more  cautiously,  as  optimistic  projections. 

The  perceptive  reader  will  notice,  however,  that  these  two  NIIRS  scales  refer  to  still  imagery, 
not  motion  imagery.  Although  there  are  analogous  video  NIIRS  (V-NIIRS)  scales,23  there  is  as 
yet  no  similarly  accepted  general  motion  image  quality  equation  to  estimate  V-NIIRS  for  either 
EO  or  IR  motion  imagery.  We  therefore  follow  the  standard  practice  of  treating  an  FMV  feed  as 

23 

Motion  Imagery  Standards  Board,  Recommended  Practice:  Video-NIIRS,  MISB  RP  0901,  October  15,  2009. 
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Figure  2.5.  EO  NIIRS  and  IR  NIIRS  Versus  GSD 
(simplified  relationships) 


a 

i/i 

2. 

bO 

O 


NIIRS 


EO 

IR 


a  series  of  still  images  and  approximating  the  V-NIIRS  of  the  FMV  feed  as  the  NIIRS  of  a  still 
image  taken  from  that  feed.  Preliminary  studies  suggest  that  this  procedure  yields  a  conservative 
estimate,  likely  underestimating  V-NIIRS  about  0.3  levels,  but  we  make  no  assumption  in  this 
regard.24 

It  should  also  be  noted  that  tracking  by  FMV  is  an  important  exploitation  task  included  on 
the  V-NIIRS  scale  that  has  no  analog  in  standard,  still-imagery  NIIRS.  For  the  analyst  to  reliably 
and  consistently  distinguish  the  target  vehicle  from  the  other  vehicles  on  the  road  using  FMV,  we 
assumed  that  the  corresponding  still  images  in  the  FMV  feed  had  to  be  of  sufficiently  high 
resolution  that  the  analyst  could  at  least  classify  the  target.25  So,  to  keep  track  of  the  vehicle  as  it 
progressed,  we  did  not  require  the  image  quality  to  be  maintained  at  such  a  high  level  that  the 
target  could  always  be  positively  identified,  but  also  we  did  not  accept  that  an  image  in  which  the 
vehicle  was  merely  a  blob  would  be  sufficient.  Instead,  we  sought  to  split  the  difference.  In  the 
absence  of  a  better  analytic  model  for  predicting  V-NIIRS  analogous  to  the  GIQE,  we  did  not 
feel  justified  in  making  any  more-refined  assumption.  (Note,  however,  that  the  rules  of 
engagement  [ROE]  we  employed  required  the  target  to  be  successfully  identified  before  the  kill.) 

Finally,  in  this  analysis,  we  did  not  model  wide-area  motion  imagery  (WAMI)  systems,  such 
as  Gorgon  Stare.  These  systems  pose  significant  challenges  for  exploitation  that  we  were  not 
prepared  to  consider  in  the  overall  study.  This  remains  an  important  area  for  future  work. 


“4  Irvine  et  al.,  2007. 

25 

Classification  is  an  intermediate  degree  of  understanding,  between  detection  and  identification. 
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Synthetic  Aperture  Radar 

Although  less  widely  used  than  the  corresponding  EO  and  IR  equations,  a  version  of  the 
GIQE  has  been  adapted  and  related  to  SAR  images,  and  there  is  a  corresponding  SAR  NIIRS 
scale.26  As  with  EO  and  IR,  SCOPEM  uses  the  GIQE  along  with  standard  SAR  NIIRS  tables  to 
estimate  which  targets  can  be  detected,  classified,  or  identified  in  a  given  SAR  image.  (Note  that 
we  do  not  model  tracking  with  SAR  because  the  SAR  systems  in  question  cannot  effectively 
image  moving  targets.27) 

At  particularly  low  grazing  angles,  however  (below  8  degrees),  this  method  does  not  capure 
the  degradation  of  SAR  imaging  well.  As  part  of  the  preceding  methodology  development  effort, 
the  SAR  model  in  SCOPEM  was  enhanced  to  employ  a  modified  GSD  that  depends  explicitly  on 
the  grazing  angle.28 

Unlike  EO  and  IR  imagery,  SAR  images  cannot  be  modeled  simply  as  snapshots — they  must 
be  built  up  over  time  as  the  platform  carrying  the  SAR  sensor  flies  to  achieve  an  angular  spread 
on  the  target.26  The  required  angular  spread  is  typically  about  2  or  3  degrees  of  parallax,  which 
typically  requires  1  or  2  minutes  of  dwell  time,  depending  on  platform  speed  and  sensor-target 
geometry.  As  Figure  2.6  illustrates,  the  dwell-time  requirements  mean  that  nearby  targets  off  to 

Figure  2.6.  Notional  Variation  of  SAR  Dwell  Time  Required  to  Image  Targets  on  the  Ground 


NOTE:  Darker  colors  represent  longer  required  dwell  times. 


“6  Ronald  G.  Driggers,  James  A.  Ratches,  Jon  C.  Leachtenauer,  and  Regina  W.  Kistner,  “Synthetic  Aperture  Radar 
Target  Acquisition  Model  Based  on  a  National  Imagery  Interpretability  Rating  Scale  to  Probability  of 
Discrimination  Conversion,”  Optical  Engineering,  Vol.  42,  No.  7,  2003. 

27 

Technically,  SAR  can  detect  very  slow-moving  vehicles,  but  in  our  model,  the  target  cither  stops  or  drives  with 
the  speed  of  traffic — the  target  vehicle  does  not  creep  along  at  one  or  two  miles  per  hour. 

“8  For  a  fuller  discussion,  see  Lingel  et  ah,  2012. 

29 

Generally  speaking,  the  target  must  be  stationary  during  the  collection  process.  Although  the  integration 
algorithms  used  to  construct  the  SAR  image  could,  in  principle,  compensate  for  known  target  motion,  this  is  rarely 
implemented,  in  part  because  the  relative  motion  of  background  with  respect  to  the  target  would  smear  the  rest  of 
the  image.  Similarly,  while  SAR  images  can,  in  principle,  be  integrated  over  an  arbitrary  flight  path,  in  practice, 
most  algorithms  require  the  collecting  platform  to  fly  in  a  straight  line  or  a  smooth  arc. 
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the  side  can  be  rapidly  collected  in  passing,  but  targets  ahead  require  much  more  time  to 
complete.30 

Ground  Moving  Target  Indicator 

As  part  of  the  methodology  development  effort  leading  up  to  this  report,  we  added  more- 
complex  models  of  GMTI  detection  and  tracking  to  SCOPEM.  Because  SCOPEM  does  not 
represent  explicit  waveform  propagation,  it  uses  expressions  at  the  macroscopic  level,  such  as 
those  derived  from  the  radar  range  equation.31  This  is  consistent  with  the  approaches  taken  for 
the  other  sensor  modalities. 

In  this  model,  the  probability  of  detection  by  the  GMTI  radar  depends  on  the  signal-to-noise 
ratio  and  the  clutter-to-noise  ratio.  The  signal-to-noise  ratio  is  estimated  from  the  strength  of  the 
radar  and  the  radar  cross  section  (RCS)  of  the  target;  the  clutter-to-noise  ratio  is  a  more- 
complicated  calculation  that  depends  on  sensor-target  geometry,  the  velocities  and  relative 
headings  of  the  sensing  platfonn  and  the  target,  and  certain  assumptions  regarding  the  RCS  of 
the  background.  A  separate  model  must  be  calibrated  to  each  specific  GMTI  sensor  for 
implementation  in  SCOPEM.  The  probability  of  maintaining  track  for  a  given  GMTI  system  is 
also  determined  offline  and  implemented  in  SCOPEM  as  a  table  of  probability  for  a  range  of 
revisit  intervals,  target  orientations,  radar  resolution,  surrounding  traffic  density,  and  many  of  the 
parameters  listed  above.32 

Environment  Models 

Environmental  factors,  such  as  terrain  and  weather,  play  a  critical  role  in  mediating  sensor- 
target  interactions.  SCOPEM  includes  a  variety  of  models  for  different  environmental  factors.  In 
this  subsection,  we  describe  the  particular  models  used  in  this  scenario:  LOS,  fog,  and  cloud 
cover. 

Line  of  Sight 

When  detennining  LOS,  the  most  basic  consideration  is  the  overall  geometry  of  the  earth. 
SEAS  natively  models  the  earth’s  spherical  geometry,  as  opposed  to  a  flat-earth  model — a 
distinction  that  is  not  always  negligible.  As  depicted  in  exaggerated  form  in  Figure  2.7,  the 
sensor  viewing  angle  (oc)  for  an  aircraft  is  always  somewhat  larger  than  the  corresponding 
grazing  angle  ((B),  a  distinction  that  a  flat-earth  model  would  ignore.  This  is  implemented  in  all 
calculations  involving  sensor-target  geometry  within  SCOPEM.  It  is  mostly  significant  for  SAR 
images  taken  at  low  grazing  angles,  where,  as  indicated  above,  the  effect  of  low  grazing  angles 


30 

If  the  dwell  is  interrupted,  we  assume  no  image  is  formed. 

31  Merrill  I.  Skolnik,  Radar  Handbook,  3rd  ed.,  New  York:  McGraw-Hill,  2008. 

32 

The  details  of  these  calculations  are  given  in  Lingel  et  ah,  2012. 
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Figure  2.7.  Spherical  Geometry 


must  be  modeled  separately.  Otherwise,  for  greater  angles,  at  the  altitudes  relevant  to  this 
scenario  (less  than  50,000  ft.),  the  difference  between  spherical  and  flat-earth  geometry  is 
small.33 

The  primary  determinant  of  LOS  between  the  sensor  and  the  target  is  their  relative  elevation 
and  the  surrounding  landscape.  Digital  terrain  elevation  data  are  used  to  define  contour  lines  at 
regular  intervals  in  SCOPEM,  which  in  turn  are  used  to  interpolate  the  altitudes  of  objects  on  the 
ground.34  These  contour  lines  also  act  as  ridgelines  that  block  LOS.  We  make  the  simplifying 
assumption  in  our  model  that  LOS  either  is  blocked  or  is  not  blocked — there  is  no  partial 
obscuration — and  we  neglect  other  features  of  the  terrain,  such  as  unusually  large  rocks  or  trees. 
For  the  mountainous,  desert  area  used  for  our  scenario,  these  assumptions  are  adequate  to 
represent  coarse  features  of  the  landscape,  but  finer  resolution  is  required  for  buildings  in  urban 
areas. 

To  represent  the  impact  of  “urban  canyons”  on  the  ability  to  track  targets,  SCOPEM  overlays 
a  separate  LOS  model  for  streets  on  top  of  the  digital  terrain  elevation  data  contour  lines 
described  above.  As  shown  in  Figure  2.8,  the  relative  height  of  the  buildings  along  a  road 


33 

When  calculating  the  GSD  for  NIIRS  estimates,  we  take  the  image  to  lie  in  a  horizontal  plane  centered  on  the 
target.  For  our  purposes,  the  curvature  of  the  earth  within  the  image  itself  is  always  negligible. 

34 

Typical  spacing  between  contour  lines  is  50  m  in  elevation. 
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Figure  2.8.  Urban  Canyon  Line-of-Sight  Model  in  SCOPEM 


compared  to  the  width  of  the  road  (both  of  which  may  be  set  at  will)  determines  the  minimum 
viewing  angle  (oc),  which  varies  with  the  obliquity  of  the  aircraft’s  viewing  angle  relative  to  the 
road  (0).  The  specific  minimum  applicable  to  any  given  situation  depends  on  the  angle  of  the 
viewing  platfonn  relative  to  the  road.  SCOPEM  allows  multiple  viewing  angles  at  intersections. 
Roads  are  assumed  to  be  two-lane  roads,  so  the  distance  from  the  target  to  the  side  of  the  road  is 
taken  to  be  (on  average)  about  one-quarter  of  the  width  of  the  road. 

Fog 

As  discussed  in  the  subsection  on  FMV  modeling,  some  effects  of  atmospheric  conditions  on 
EO  and  IR  image  quality  must  be  modeled  separately.  We  use  the  Air  Force  Research 
Laboratory  program  MODTRAN  to  determine  the  approximate  transmittance  of  EO  and  IR 
radiation  through  different  atmospheres,  as  a  function  of  range  and  viewing  angles. 15 

To  model  the  effects  of  fog  or  haze  on  image  quality,  the  transmittance  calculated  in  this 
manner  is  used  to  rescale  the  signal-to-noise  ratio  in  the  GIQE,  one  of  the  several  parameters  for 
which  we  previously  used  only  the  average  value.  The  scaling  factor  is  the  ratio  of  the 
transmittance  through  fog  to  the  transmittance  on  a  clear  day  for  the  same  range  and  viewing 
angle.  (A  ratio  of  zero  would  block  all  radiation;  a  ratio  of  one  would  yield  no  effect.)  The  result 


35 

The  particular  instantiation  of  MODTRAN  that  RAND  used  for  this  effort  is  PCWinMod,  Versions  4  and  5. 
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is  a  significant  degradation  of  image  quality  for  many  EO  systems,  but  less  so  for  IR  systems, 
because  these  wavelengths  have  greater  ability  to  penetrate.36 

Cloud  Cover 

SCOPEM  implements  a  simplistic  model  of  cloud  cover  designed  to  capture  one  particular 
effect:  restricted  LOS  to  ground  targets  when  flying  above  the  cloud  ceiling.  As  before,  we 
assume  LOS  is  either  completely  clear  or  totally  blocked  at  any  given  moment  (i.e.,  the  clouds 
are  completely  opaque)  although  the  cloud  cover  above  any  given  point  will  change  over  time. 
We  model  a  single  layer  of  clouds  only:  There  is  little  advantage  to  modeling  additional  layers 
when  using  this  simple  of  a  representation. 

To  model  the  cloud  layer,  we  look  to  Air  Force  Combat  Climatology  Center  data  to  estimate 
the  typical  percentage  of  cloud  coverage  in  a  given  region  during  a  given  season — and,  where 
available,  the  average  spatial  and  temporal  coherence  of  that  cloud  cover.  We  divide  the  sky  into 
a  grid  and  populate  each  square  as  either  clear  or  obscure,  based  on  the  typical  percentage  of 
cloud  cover.37  The  spatial  coherence  is  used  to  detennine  the  spacing  of  the  grid;  the  temporal 
coherence  is  used  to  determine  how  often  the  grid  must  be  refreshed. 

Note  that,  while  we  use  a  random  draw  to  determine  which  squares  are  obscured,  each  square 
remains  either  clear  or  obscured  until  redrawn.  We  do  not  make  a  separate  random  draw  for  each 
viewing  attempt  because  that  would  unrealistically  allow  a  sensor  to  penetrate  a  patch  of  cloud 
merely  by  looking  repeatedly.  It  also  assures  that  all  sensors  in  the  region  face  the  same  pattern 
of  obscuration.  An  aircraft  can,  however,  see  under  a  patch  of  cloud  by  obtaining  a  favorable 
viewing  angle  through  a  nearby  hole. 

Note  that  we  do  not  model  the  effects  of  cloud  cover  on  radar  and  do  not  model  the  effects  of 
rainfall.  (“Rain  fading”  is  known  to  affect  radar  returns.38)  We  consider  only  partly  cloudy 
weather  without  rain  or  other  precipitation.  Overcast  skies  would  make  the  mission  as  defined 
here  impossible  for  platfonns  flying  above  the  cloud  ceiling  because  the  target  could  not  be 
imaged  with  the  quality  necessary  to  identify  it,  while  any  significant  precipitation  would  hamper 
operations  beneath.  A  more-sophisticated  model  of  RPA  operations  would  be  required  to  model 
the  hunter-killer  mission  under  such  conditions. 


36  We  considered  transmittance  effects  for  medium-wavelength  IR  radiation  (approximately  3  to  8  pm). 

37 

We  use  latitude  and  longitude  to  define  the  grid,  which  is  therefore  not  strictly  square,  but  narrower  at  higher 
latitudes.  For  the  size  of  region  modeled  here,  this  variation  is  negligible. 

38  Skolnik,  2008. 
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3.  The  Hunter-Killer  Mission 


Introduction 

The  hunter-killer  mission  is  understood  best  not  as  a  specific  mission  but  as  a  class  of 
missions  in  which  aircraft  hunt  for  and  ultimately  kill  a  specific  individual  target  on  the  ground. 
One  or  more  aircraft  may  participate  in  the  search;  the  search  area  may  be  large  or  small;  the 
target  may  be  mobile  or  stationary  or  may  exhibit  more-complex  behaviors;  the  ROE  may  be 
loose  or  restrictive;  and  the  environmental  conditions  can  vary  greatly. 

The  class  of  missions  is  too  broad  for  us  to  explore  fully.  Instead,  we  selected  one  particular 
scenario  so  that  we  could  explore  its  variants.  We  have  chosen  to  examine  a  specific  target  in  a 
specific  geographical  area  while  varying  the  number  and  type  of  aircraft,  the  ROE,  and  the 
environmental  conditions.  These  choices  allow  us  to  explore  certain  basic  considerations  for 
RPA  design  concretely,  but  the  analysis  is  far  from  exhaustive.  Other  scenarios  would  illuminate 
other  trade-offs. 

We  selected  the  particular  scenario  that  we  did  to  draw  out  the  differences  in  mission 
effectiveness  between  the  particular  RPA  concepts  we  studied.  It  would  not  have  been  difficult 
to  select  a  scenario  in  which  all  the  RPA  concepts  we  considered  always  succeeded  or  in  which 
they  never  succeeded — but  doing  so  would  not  have  illuminated  the  trade-offs  among  them.  We 
adjusted  the  CONOPS  for  each  RPA  concept  to  represent,  as  best  we  could,  its  intelligent 
employment  in  the  hunter-killer  mission. 

Although  the  scenario  we  modeled  is  representative  of  many  hunter-killer  missions  as  they 
are  executed  today,  it  is  not  representative  of  others.  It  also  relies  on  several  crucial  assumptions. 
Most  important,  the  threat  environment  we  modeled  is  permissive,  meaning  that  there  is 
negligible  threat  to  the  aircraft.  This  permits  the  use  of  CONOPS  untempered  by  survivability 
concerns. 

In  the  particular  scenario  we  considered,  the  initial  cue  for  the  target  vehicle  is  urgent  but 
imprecise,  allowing  only  that  the  enemy  is  expected  to  emerge  from  hiding  somewhere  in  a 
5  nmi2  area  within  15  minutes.  The  target’s  ultimate  destination  is  believed  to  be  outside  of  this 
area  but  is  otherwise  unknown.  Finding  the  target  vehicle  before  it  leaves  the  area  therefore 
necessitates  a  rapid,  brute-force  search. 

Because  the  area  is  not  especially  large,  the  search  is  not  difficult,  but  environmental  factors 
complicate  it.  The  cued  area  is  an  urban  environment,  which  constrains  LOS  to  steep  viewing 
angles.  Moreover,  in  many  of  the  variants  we  considered,  the  RPA  sensor  operator  is  forced  to 
contend  with  other  effects  detrimental  to  establishing  and  maintaining  LOS,  including  fog  and 
clouds.  The  parameters  of  the  initial  cue,  as  described  above,  were  selected  to  highlight  how 
these  environmental  conditions  affect  operational  effectiveness. 
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The  ROE  and  CONOPS  we  selected  are  also  fairly  restrictive.  They  require  that  the  target  be 
positively  identified  with  FMV  and  that  the  RPA  sensor  operator  keep  the  target  in  view  with 
FMV  whenever  weather  conditions  permit.  This  places  a  high  burden  on  the  required  resolution 
of  the  imagery,  which  must  yield  an  estimated  NIIRS  quality  level  of  7.0  or  8.0. 39  (Both  cases 
are  modeled.)  While  reflective  of  current  practice,  these  requirements  effectively  relegate  GMTI 
to  a  lesser  role  than  it  could  otherwise  play.  These  requirements  also  preclude  the  RPA  from 
flying  at  higher  altitudes,  which  in  turn  reduces  the  effective  ground  coverage  area  that  the  RPA 
sensor  operator  can  achieve.  The  ROE  therefore  force  the  RPA  pilot  to  choose  a  flight  profile 
that  balances  the  competing  demands  for  high  resolution  and  greater  ground  coverage  area. 

We  made  several  simplifying  operating  assumptions  as  well.  We  did  not  consider  the 
possibility  of  communications  jamming  or  loss  of  Global  Positioning  System  connectivity,  either 
of  which  could  be  potentially  devastating.  We  assumed  that  each  RPA  had  sufficient 
communications  capacity  and  PED  resources  to  fulfill  its  mission — a  necessary  but  admittedly 
heroic  assumption  mandated  largely  by  certain  technical  limitations.  Finally,  while  the  enemy  is 
allowed  some  awareness  that  it  is  in  danger  of  being  watched,  we  assumed  that  it  could  neither 
see  nor  hear  the  RPA.  The  target  vehicle  also  does  not  employ  camouflage,  concealment,  or  any 
of  the  other  clever  ruses  that  have  at  times  been  observed  in  the  field:  The  only  adaptive  enemy 
behavior  is  to  come  to  a  halt  from  time  to  time,  in  an  attempt  to  confuse  possible  GMTI  tracking. 

These  choices  and  assumptions  enabled  us  to  avoid  certain  issues  so  that  we  could  focus 
instead  on  the  broader  trade-offs  between  platform  size  and  platform  number;  differing  sensor 
performance;  and  the  complicating  effects  of  darkness,  fog,  and  cloud  cover.  However,  they  also 
inherently  limit  the  applicability  of  our  findings  on  those  trade-offs.  Nevertheless,  we  believe 
that  our  analysis  illuminates  some  important  basic  considerations  in  RPA  design  choices.  It  also 
demonstrates  a  successful  analytical  methodology  that  may  be  expanded  to  address  more- 
challenging  scenarios. 

Sensor  Capabilities 

The  sensor  capabilities  we  considered  drive  the  particular  CONOPS  for  each  RPA  design 
concept  to  a  considerable  degree.  Four  sensors  play  a  role  in  this  analysis:  the  Multispectral 
Targeting  System,  Model  B  (MTS-B);  Raven  Eye  I;  Raven  Eye  II;  and  Lynx.  The  first  three  are 
FMV  sensors,  with  both  EO  and  IR  collection  capabilities.  Lynx  is  a  radar  system  with  SAR  and 
GMTI.  The  capabilities  of  these  sensors  are  described  here. 

Multispectral  Targeting  System,  Model  B 

The  MTS-B  has  EO  (daytime  color)  and  IR  (day-and-night  monochrome)  FMV  capabilities. 
Each  of  these  has  several  fixed  modes  of  magnification  or  field  of  view  (FOV),  as  given  in 


39 


For  a  description  of  the  NIIRS  scale,  see  Leachtenauer  et  al.,  1997. 
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Table  3.1.  For  our  purposes,  we  have  grayed  out  the  “Ultranarrow”  mode  because  it  is  an 
electronic  zoom  of  the  “Narrow”  mode,  rather  than  a  higher  sensor  resolution.  All  modes  render 
FMV  in  480  x  640  pixels,  and  the  corresponding  FOVs  are  always  in  that  same  3:4  proportion. 

The  field  of  regard  for  MTS-B  effectively  extends  360  degrees  in  azimuth  and  down  to  90 
degrees  in  elevation  (straight  down),  but  in  practice,  such  viewing  angles  are  discouraged 
because  they  increase  likelihood  of  losing  the  target.  The  short  but  nonnegligible  latency  inherent 
in  controlling  the  RPA  remotely  implies  that  the  sensor  operator  should  “lead”  the  target  it  is 
attempting  to  track,  which  is  more  difficult  in  such  cases.  Note  that  the  MTS-B  does  not  have 
continuous  zoom  capabilities  but  allows  only  discrete  modes,  as  shown  in  Table  3.1. 

Raven  Eye  I 

Raven  Eye  I  is  a  lighter  sensor  package,  with  less  resolution  in  its  modes.  The  specifications 
of  the  various  modes  are  given  in  Table  3.2.  Note  that  the  EO  system  (color  television)  has 
continuous  zoom  between  the  limits  given,  while  the  IR  mode  has  fixed  levels  of  magnification. 
The  ability  to  zoom  continuously  in  the  EO  mode  allows  the  platform  to  sit  in  a  “sweet  spot”  for 
the  hunter-killer  mission  in  nearly  all  cases,  something  not  always  possible  for  other  sensors.  The 
field  of  regard  is,  for  our  purposes,  not  limited. 


Table  3.1.  MTS-B  Modes, 
Field  of  View,  and  Pixels 


Mode 

EO 

(degrees) 

IR 

(degrees) 

Ultrawide 

34  x  45 

34  x  45 

Wide 

17  x  22 

17  x  22 

Medium 

5.7  x  7.6 

5.7  x  7.6 

Narrow-Medium 

0.96  x  1.3 

2.8  x  3.7 

Narrow 

0.16  x  0.22 

0.47  x  0.63 

Ultranarrow 

0.08  x  0.11 

0.23  x  0.31 

SOURCE:  Bart  E.  Bennett,  Joel  Kvitky,  Daniel  M. 
Norton,  and  Michael  Nixon,  MQ-1  Predator  and 
MQ-9  Reaper  Air-to-Ground  Sensor  Capabilities, 
Santa  Monica,  Calif.:  RAND  Corporation,  TR- 
1 1 74-AF,  2013,  Not  available  to  the  general 
public,  Table  8. 

NOTE:  Dimensions  have  been  rounded  to  two 
significant  figures. 
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Table  3.2.  Raven  Eye  I  Capabilities 


Mode 

EO  (480  x  640) 

(degrees) 

IR  (320  x  240) 

(degrees) 

Maximum  (wide) 

20.25  x  27.0 

16  x  22 

Medium 

i 

0.64  x  0.85 

5.2  x  6.9 

Minimum  (narrow) 

1.3  x  1.7 

Raven  Eye  II 

Raven  Eye  II  is  similar  to  Raven  Eye  I,  except  that  it  has  smaller  FOVs  and  correspondingly 
higher  resolution  (Table  3.3).  It  is  also  three  times  heavier  (110  lbs.  versus  35  lbs.)  because  of  its 
larger  aperture.40  Both  aspects  play  a  role  in  the  hunter-killer  mission  we  simulated.  With  Raven 
Eye  II,  and  RPA  can  achieve  the  required  NIIRS  at  a  higher  altitude,  but  the  search  area  it  can 
cover  is  likely  smaller.  As  with  Raven  Eye  I,  the  field  of  regard  is  effectively  not  limited. 


Table  3.3.  Raven  Eye  II  Capabilities 


Mode 

EO  (480  x  640) 

(degrees) 

IR  (256  x  256) 

(degrees) 

Maximum  (wide) 

10.2  x  13.6 

13.5  x  13.5 

Medium 

1 

0.3  x  0.4 

2.9  x  2.9 

Minimum  (narrow) 

0.73  x  0.73 

Lynx 

Lynx  provides  both  SAR  and  GMTI  capabilities.41  This  is  the  system  on  the  Reaper  today, 
although,  in  practice,  the  SAR  mode  is  used  infrequently,  and  its  GMTI  capabilities  are  currently 
exploited  very  little  if  at  all.  The  Lynx  SAR  mode  has  270  degrees  azimuthal  coverage;  the  rear 
quarter  is  not  in  the  field  of  regard.  The  Lynx  GMTI  mode  is  more  limited,  with  only  90  degrees 
azimuthal  coverage — only  the  front  quarter.  Both  modes  are  restricted  in  field  of  regard  to 
viewing  angles  lower  60  degrees  (the  nadir  is  not  visible).  These  limitations  are  important 
because  they  can  interfere  with  the  ability  to  track  or  image  the  target  under  some  situations. 
Although  the  Lynx  SAR  has  several  spot-mode  resolutions,  we  assume  the  highest  resolution  is 
used  for  imaging  because  this  is  necessary  to  obtain  a  suitable  image  for  this  mission. 


40 

Northrop  Grumman  Corporation,  “Raven  Eye  I:  Unmanned  Multimission  Stabilized  Payload,”  factsheet,  Falls 
Church,  Va.  DS-295-BAS-0803,  undated  a;  Northrop  Grumman  Corporation,  “Raven  Eye  II:  Unmanned  Multi¬ 
mission  Stabilized  Payload,”  factsheet,  Falls  Church,  Va.  DS-296-BAS-0803,  undated  b. 

41  We  simulated  AN/APY-8,  Lynx  Block  20. 
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Specific  Scenario 

In  our  scenario,  prior  intelligence  has  indicated  that  the  target  is  likely  to  emerge  from  hiding 
somewhere  within  an  urban  area  of  approximately  5.0  nmi2  (approximately  17  km2)  within  a 
particular  window  of  time  (about  15  min.).  We  made  the  simplifying  assumption  that  the  time 
window  indicated  by  the  cue  is  sufficiently  circumscribed  and  the  warning  sufficiently  advanced 
that  the  RPA  may  arrive  on  station  prior  to  the  window  and  may  remain  on  station  throughout  its 
duration.42  As  we  framed  the  problem,  increased  range  and  endurance  would  permit  the  RPA  to 
prosecute  effectively  a  larger  set  of  the  hunter-killer  class  of  missions  (e.g.,  longer  time 
windows,  or  a  farther  distance  from  the  base)  than  RP  As  of  lesser  range  and  endurance,  but  these 
attributes  would  not  affect  operational  effectiveness  once  on  station.43  This  choice  levels  the 
playing  field  somewhat  between  the  smaller  and  larger  platforms  for  the  purposes  of  comparison 
in  this  scenario.  Relaxing  this  assumption  would  make  the  smaller  RPAs  less  viable  to  the 
mission  as  a  whole. 

We  used  a  small  urban  area  near  Afghanistan  for  scaling  the  landscape,  the  size  of  the  city, 
and  some  of  the  road  structure  and  as  a  reference  point  for  typical  weather  conditions  (see  later 
discussion  of  flight  profiles.)  The  urban  area  is  not  especially  dense  but  is  characterized  by  roads 
that  are  somewhat  narrower  than  the  heights  of  the  buildings,  creating  an  “urban  canyon”  effect. 
In  the  narrowest  of  these  canyons,  depending  on  the  direction  of  the  aircraft  relative  to  the  roads, 
the  minimum  required  grazing  angle  could  be  as  high  as  63  degrees;  shallower  angles  could 
leave  portions  of  the  road  obscured.44  In  most  cases,  however,  the  minimum  viewing  angle  was 
less  restrictive. 

Once  on  station,  the  RPA  conducts  a  raster  search  for  the  target  using  FMV.  (In  a  raster 
search,  the  sensor  sweeps  out  a  pattern  back  and  forth  in  alternating  rows,  as  a  farmer  might 
plow  a  field.)  As  will  be  discussed  in  the  next  subsection,  the  exact  parameters  of  this  raster 
search  will  vary  based  on  environment,  weather,  and  platform  capabilities.  The  general  scheme, 
however,  is  always  the  same:  FMV  sensors  collect  in  a  wide  mode  to  detect  vehicles  and 
occasionally  zoom  in  to  a  narrow  mode  to  attempt  to  identify  them. 

An  important  alternative  CONOPS  that  we  did  not  explore  would  have  been  to  use  GMTI  to 
cue  the  narrow  identification  mode  of  the  FMV  sensor.  There  are  many  variants  of  the  hunter- 
killer  scenario  in  which  this  CONOPS  is  likely  preferable  to  a  raster  search,  such  as  those  that 
afford  a  better  initial  cue  or  a  longer  timeline.  The  particular  scenario  we  chose,  however,  was 
not  favorable  to  demonstrate  this  CONOPS.  Because  the  ROE  demanded  positive  identification 


42 

“  Relaxing  this  requirement  would  reduce  all  measures  by  whatever  fraction  of  the  time  window  was  missed. 

43 

Increased  speed  can  improve  operational  effectiveness  when  on  station,  however,  and  this  emerges  from  the 
subsequent  analysis. 

44 

We  use  grazing  angle  to  mean  the  angle  between  the  sensor-target  line  and  the  horizontal  plane  of  the  target  and 
viewing  angle  to  mean  the  angle  between  the  sensor-target  line  and  the  horizontal  plane  of  the  sensor.  As  indicated 
earlier,  these  differ  in  more-realistic  spherical  geometry. 
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by  FMV,  because  the  NIIRS  requirements  for  that  identification  were  fairly  strict,  and  because 
the  vehicle  emerged  within  an  urban  canyon  that  impaired  LOS,  the  RPA  was  forced  to  fly  closer 
to  the  ground  and  had  more  limited  viewing  angles  than  it  otherwise  would  have  enjoyed — all  of 
which  contributed  to  a  significantly  reduced  effective  ground  coverage  area.  For  this  reason,  the 
RPA  sensor  operator  would  only  have  been  able  to  take  very  limited  advantage  of  the  wider  field 
of  regard  that  GMTI  could  provide.46  Thus,  we  did  not  use  GMTI  to  its  fullest  possible 
advantage  in  this  analysis:  We  left  it  in  a  supporting  role  to  FMV.  Although  this  does,  however 
unfortunately,  reflect  how  the  Reaper  is  most  commonly  employed  today  in  similar  hunter-killer 
missions — a  fact  which  we  intentionally  captured — it  does  not  accurately  reflect  the  full  potential 
of  GMTI  to  contribute  to  this  type  mission,  especially  as  part  of  a  more-advanced  future  RPA 
concept.  This  is  an  area  of  research  that  should  be  further  explored. 

In  this  scenario,  the  target  rides  in  a  ground  vehicle,  either  a  small  or  large  passenger  truck.46 
At  some  random  time,  the  vehicle  will  indeed  emerge  from  a  random  hiding  place  and  head  to 
one  of  several  possible  destinations  in  a  more  rural  area.47  (We  do  not  allow  the  RPA  to  know  in 
advance  where  the  target  is  ultimately  headed.).  The  possible  starting  locations  are  shown  in 
Figure  3. 1 ,48  The  target  will  stop  at  irregular  intervals  in  an  attempt  to  elude  pursuit,  but 
otherwise  moves  with  the  speed  of  traffic  on  the  road  and  employs  neither  camouflage  nor 
concealment  techniques.  We  assume  there  is  one  enemy  vehicle  only. 

Figure  3.1.  Initial  Possible 
Starting  Locations 
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45 

Attempting  to  identify  all  the  targets  in  significantly  a  larger  field  of  regard  would  also  have  required  us  to  revisit 
our  heroic  assumption  of  sufficient  PED. 

46  2 

Modeled  as  approximately  a  3  m  radar  cross  section. 

47 

For  simplicity,  we  assumed  that  the  entire  time  window  is  cither  daylight  or  nighttime.  This  allowed  us  to 
distinguish  more  clearly  the  effects  of  EO  versus  IR  sensors. 

48 

For  each  Monte  Carlo  run,  the  starting  and  destination  points  are  randomized;  however,  for  any  given  pair  of 
points,  the  vehicle  will  always  follow  the  same  route  from  one  to  the  other. 
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If  and  when  the  RPA  or  RPAs  locate  the  target,  they  will  attempt  to  track  it  through  the  urban 
canyon  to  a  suitable  location  outside  the  city,  typically  5  to  10  nmi  away  (depending  on  the  final 
destination),  where  the  ROE  pennit  attempting  to  kill  the  target.49  Where  available,  onboard 
GMTI  will  be  used  to  track  vehicles,  and  SAR  may  also  be  used  if  the  vehicle  stops  and  weather 
conditions  make  FMV  ineffective.  If  the  RPA  loses  track  of  the  vehicle,  it  will  attempt  to 
reacquire  the  track.  (We  assume  it  must  also  reidentify  the  vehicle  in  that  case.)  A  representative 
screenshot  of  the  graphical  output  of  SEAS  running  a  hunter-killer  scenario  is  shown  in 
Figure  3.2. 

For  multiple  RPAs,  tracking  is  cooperative:  Each  RPA’s  target  sightings  are  sent  to  the 
others.  Although  these  sightings  are  fused  locally  within  each  platform,  so  that  there  may  be  very 
minor  differences,  all  RPAs  have  nearly  the  same  operational  picture  in  practice.  We  assumed 
that  the  target  sightings  shared  in  this  manner  are  100  percent  correlated,  meaning  that  all  the 
RPAs  are  indeed  attempting  to  track  the  same  vehicle  at  the  same  time.  Jamming  and  other 
potential  communications  difficulties  are  not  considered.  The  tracking  time  required  depended 
on  the  specific  circumstances  of  when  and  where  the  target  was  identified  and  which  destination 
it  was  headed  toward,  but  typically  the  target  needed  to  be  tracked  for  about  15-20  minutes  to 
reach  the  kill  zone.  Ambient  traffic  was  not  explicitly  modeled. 

Figure  3.2.  Representative  SEAS/SCOPEM 
Screen  Shot 


49 

The  initial  search  area  is  sufficiently  constrained  that,  in  effect,  target  must  be  detected  and  identified  within  the 
urban  canyon. 
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Flight  Profiles 

Choosing  the  right  flight  profile  for  each  RPA  concept  is  crucial  to  success  in  the  hunter- 
killer  scenario  we  modeled.  Within  the  context  of  the  FMV  raster  search  in  particular,  we  tuned 
the  flight  profiles  separately  for  each  RPA  concept,  sensor  package,  and  weather  condition  to 
achieve  the  best  mission  outcomes  under  the  circumstances.  In  a  handful  of  cases,  the  flight 
profile  changes  partway  through  the  mission,  usually  shifting  to  a  higher  altitude  once  the  target 
has  been  identified  and  the  task  switches  from  searching  to  following.  Although  changing  the 
search  pattern  in  this  manner  could  make  certain  types  of  comparison  more  difficult,  this  form  of 
(limited)  optimization  reflects  more  accurately  how  an  RPA  pilot  would  actually  conduct  this 
kind  of  mission.  Therefore,  we  believe  this  gives  us  better  insight  into  the  operational 
effectiveness  achievable  under  the  given  conditions  with  these  RPA  concepts. 

The  key  to  determining  the  flight  profile  in  most  cases  was  selecting  the  initial  flight  altitude. 
To  find  and  detect  the  target,  the  sensor  operator  uses  a  wider  FMV  mode,  with  lower  resolution; 
identifying  the  target  requires  a  narrower  FMV  mode  with  higher  resolution.  Ideally,  the  RPA 
would  fly  as  high  as  possible  to  cover  as  much  area  as  possible  at  one  time  (and  possibly 
improve  survivability  as  well)  and  still  be  able  to  perform  both  the  detection  and  identification 
objectives,  but  there  is  an  inherent  tension  between  the  two  objectives.  It  was  something  of  a 
surprise  that  the  most  stringent  NIIRS  requirement,  positive  identification  of  the  target,  was  not 
always  the  limiting  factor  in  operational  effectiveness.  This  is  because  the  FMV  modes  typically 
did  not  have  continuous  zoom  but  allowed  only  fixed  levels  of  magnification.  Thus,  for  example, 
in  some  cases  it  proved  advantageous  to  fly  at  a  somewhat  lower  altitude  to  take  advantage  of  the 
wider  FOV  for  detection  that  would  otherwise  be  unable  to  provide  the  necessary  image  quality. 

Because  the  NIIRS  requirements  we  imposed  on  the  mission  were  central  to  the  mission 
design  and  outcome,  we  modeled  two  different  cases.  In  the  first  case,  we  required  NIIRS  8.0  for 
identification  for  both  EO  and  IR  sensors  and  NIIRS  4.0  for  detection.  In  the  second  case,  we 
considered  alternative  requirements  of  NIIRS  7.0  for  identification,  NIIRS  3.0  for  detection  by 
EO,  and  NIIRS  4.0  for  detection  by  IR.50  (Henceforth,  we  will  use  “NIIRS  8.0”  and  “NIIRS  7.0” 
as  shorthand  for  these  two  sets  of  requirements,  respectively.)  In  both  cases,  we  used  NIIRS  6.0 
as  the  threshold  for  the  RPA  sensor  operator  being  able  to  distinguish  the  vehicle  reliably  from 
nearby  confusers  for  the  purpose  of  following  it  with  FMV.51  For  SAR,  we  set  a  requirement  of 
NIIRS  3.0  for  detection  but  set  no  requirement  for  identification,  assuming  that  SAR  alone 
would  be  incapable  of  satisfying  the  ROE  for  killing  the  target  under  these  conditions.  In  all 
cases,  EO  sensors  were  used  in  daytime,  IR  sensors  at  night. 


50  These  are  reasonable  values  for  vehicles  and  trucks.  (For  example,  see  Leachtenauer,  1997.)  Given  the  primacy  of 
FMV  in  this  scenario,  they  may  be  considered  the  defining  characteristics  of  these  targets  in  our  modeling,  for  all 
intents  and  purposes. 

51  NIIRS  6.0  was  specified  as  the  requirement  for  classification,  although  it  played  little  role  in  the  analysis. 
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We  determined  that,  to  be  most  effective,  the  smaller  RPA  concepts,  carrying  only  Raven 
Eye  I  or  Raven  Eye  II,  needed  to  fly  below  the  cloud  ceiling.  The  larger  RPA  concepts,  carrying 
the  MTS-B/Lynx  combination,  however,  were  more  effective  flying  above  the  cloud  ceiling 
because  the  GMTI  radar  could  often  maintain  track  when  the  vehicle  went  behind  a  cloud.  This 
determination  does  depend  on  the  nature  of  the  cloud  cover,  however.  In  our  scenario,  the  cloud 
ceiling  was  taken  to  be  3,000  ft.,  with  40  percent  cloud  cover,  consistent  with  the  geographic 
area. 

Table  3.4  summarizes  the  final  altitudes  for  the  model.  The  cloud  cover  parameters  are  as 
described  above.  For  the  case  of  fog,  the  transmittance  of  fog  we  modeled  for  visible  (EO) 
wavelengths  was  taken  to  be  approximately  0.125,  and  the  transmittance  for  IR  wavelengths  was 
0.75.  (However  note  that  these  transmittances  varied  dramatically  depending  on  viewing  angle, 
ranging  from  0.05  to  over  0.90  depending  on  the  circumstances.  These  figures  are  for  the  default 
45-degree  viewing  angle  we  took  for  FMV  detection  and  tracking  during  the  raster  search.  See 
earlier  discussion  on  fog  for  explanation  of  transmittance.) 

It  is  also  worth  observing  that  the  MTS-B,  which  offers  only  discrete  viewing  modes,  did  not 
always  permit  flight  profiles  that  hit  the  “sweet  spot”  as  well  as  the  Raven  Eye  I  and  II,  which 
offer  continuous  zoom  capability.  This  capability  allows  the  RPA  sensor  operator  to  balance  the 
competing  demands  for  higher  resolution  and  greater  ground  coverage  at  will. 

For  multiple  platforms,  we  were  careful  to  stagger  the  timing  to  take  maximum  advantage  of 
the  search  pattern.  Once  the  vehicle  was  identified,  the  RPAs  circled  around  the  vehicle  to  keep 
it  in  view  as  they  tracked  it.  The  size  of  the  circle  was  adjusted  to  keep  the  average  viewing  angle 
at  approximately  45  degrees.  Multiple  platforms  followed  the  same  circle  but  staggered  their 
angular  spacing  to  take  advantage  of  different  simultaneous  vantage  points — this  permitted  them 
to  keep  the  vehicle  in  view  better  in  the  urban  canyon.  For  three  vehicles,  we  chose  even  angular 
spacing  (120-degree  separation).  For  two  vehicles,  however,  we  chose  uneven  angular  spacing 


Table  3.4.  Altitudes  (ft.  above  ground  level) 


Sensors 

Time 

NIIRS  7.0 

NIIRS  8.0 

Clear 

Fog 

Cloud 

Clear 

Fog 

Cloud 

MTS-B  /  Lynx 

Day 

16,000 

11,400 

16,000 

8,000 

5,700 

8,000 

Night 

11,300 

11,100 

11,300 

8,550 

8,400 

8,550 

Raven  Eye  1 

Day 

8,900 

6,350 

3,000 

4,500 

3,200 

3,000 

Night 

1,975 

1,950 

1,975 

1,550 

1,500 

1,550 

Raven  Eye  II 

Day 

19,000 

13,500 

3,000 

9,500 

6,750 

3,000 

Night 

2,550 

2,500 

2,550 

2,550 

2,500 

2,550 

NOTES:  Blue  cells  indicate  that  the  resolution  or  FOV  of  the  detection  mode  was  the  limiting  factor. 
Yellow  cells  indicate  that  the  resolution  of  the  identification  mode  was  the  limiting  factor.  Gray  cells 
indicate  that  cloud  ceiling  was  the  limiting  factor. 
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(120  degrees  and  240  degrees).  Even  spacing  (180-degree  separation)  would  place  the  RPAs  on 
exact  opposite  sides  of  the  circle,  which  proved  suboptimal  because  they  were  both  vulnerable  to 
losing  the  target  in  the  urban  canyon  at  the  same  time. 

Killing  the  Target 

As  mentioned  earlier,  we  did  not  simulate  weapons  within  SCOPEM.  In  our  model,  if  the 
target  could  be  successfully  found,  identified,  tracked  to  a  designated  kill  zone,  and  maintained 
continuously  in  track  long  enough,  then  the  target  could  be  killed.  For  our  purposes,  the  key 
question  is  how  much  time  is  “long  enough”  to  complete  the  strike.  This  depends  on  the 
CONOPS  for  killing  the  target,  which  in  turn  depends  on  whether  any  of  the  RPAs  prosecuting 
the  mission  are  carrying  weapons  or  need  to  call  in  another  aircraft  to  do  so.  It  also  depends  on 
the  command  and  control  process  that  leads  up  to  the  decision  to  confirm  the  strike.  In  all  cases, 
we  relied  on  the  assumption  that  the  RPA(s)  can  communicate  effectively  with  the  combined  air 
operations  center  during  the  “kill”  portion  of  the  hunter-killer  mission.  We  looked  at  the  two 
CONOPS  described  next. 

Case  1:  RPA  Strike 

The  most  straightforward  case  is  simply  for  the  RPAs  that  hunt  the  target  to  kill  it  as  well. 
This  is  usually  how  the  hunter-killer  mission  is  envisioned.  This  may  not  be  possible  for  smaller 
RPAs  concepts,  but  for  the  larger  ones,  like  the  MQ-9,  it  certainly  is.  In  this  case,  the  time  lines 
are  short.  Once  the  target  has  been  validated,  meaning  that  the  ROE  are  satisfied,  the  RPA 
maneuvers  to  employ  weapons  and  then  fires.52  (Afterward,  it  presumably  performs  a  postattack 
assessment.)  Elapsed  time  from  track  to  kill  is  50  seconds  or  less.  Depending  on  the  weapon 
used,  it  may  be  necessary  for  the  RPA  to  remain  in  visual  contact  with  the  target  throughout  the 
entire  process.  The  situation  is  shown  in  Figure  3.3. 

As  indicated  earlier,  to  reach  the  kill  zone,  the  time  for  which  the  target  had  to  be  tracked 
varied  considerably,  but  1 5  to  20  min.  was  typical  (and,  within  the  model,  the  RPA  usually 
retained  track  on  the  target  for  considerably  longer  than  this).  Given  the  somewhat  arbitrary 
nature  of  the  location  of  the  “kill  zone”  and  the  “end”  of  the  scenario  and  given  the  very  short 
time  line  required  for  the  strike  itself,  if  the  RPA  was  able  to  maintain  track  all  the  way  to  the 
“end”  of  the  scenario,  we  assumed  it  had  sufficient  time  to  complete  the  firing  process. 


52 

We  assume  the  RPA  pilot  has  “on  scene”  commander  authority  to  engage  targets  that  meet  the  commander’s 
guidance  and  ROE  restrictions. 
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Figure  3.3.  Single  RPA  Strike,  Firing  Timeline  (times  in  seconds) 
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Case  2:  Call  for  Fires 

A  more-challenging  case  involves  the  situation  in  which  the  RPA  does  not  have  the 
necessary  weaponry  to  complete  the  strike.  This  may  be  because,  as  for  Group  3  (and  possibly 
Group  4),  the  airframe  is  too  small  or  too  underpowered  to  carry  weapons  in  addition  to  the 
(large)  sensor  package.  It  may  also  be  that  there  are  different  versions  of  the  RPA — some  with 
only  sensors,  some  with  only  weapons — and  a  weaponized  version  must  be  pulled  away  from 
another  task  to  complete  the  kill.  This  case  is  only  applicable  when  basing  is  available  and 
relatively  close  and  when  the  United  States  has  good  access  to  the  airspace  for  its  strike  aircraft. 

In  this  situation,  all  the  times  described  in  the  previous  case  also  apply,  plus  an  additional  5 
sec.  must  be  taken  to  transfer  (digitally)  the  target  geolocation  data  to  the  strike  platform.  (Unlike 
in  Case  1 ,  it  was  not  previously  tracking  the  target.)  Of  course,  adding  a  few  more  seconds 
makes  no  difference  in  our  model.  The  real  challenge  in  terms  of  time  comes  from  two  sources: 
the  command  and  control  processes  necessary  to  retask  the  strike  platform  from  some  other 
mission  and  the  flight  time  required  for  the  strike  platform  to  arrive  on  station. 

We  assumed  that  the  target  is  of  high  value  and  that  the  Air  Force  will  in  fact  treat  it  as  a 
time-sensitive  target.5  ’  We  also  assumed  that  a  strike  aircraft  is  already  on  call  and  available  to 
be  retasked  and  that  the  commander  at  the  combined  air  operations  center  has  the  authority  to 
retask  it  without  additional  coordination.  Even  under  these  favorable  conditions,  however,  it  can 
take  up  to  5  min.  to  process  the  retasking  request,  up  to  5  more  min.  to  actually  retask  the 
aircraft;  the  aircraft  still  has  to  fly  to  arrive  at  its  new  station.  Thus,  it  may  take  up  to  10  min.  to 
retask,  plus  flight  time.  The  distance  that  the  strike  platform  must  fly  is  the  governing  factor  in 
whether  this  CONOPS  is  possible,  of  course. 

Because  the  times  involved  are  comparable  to  those  in  the  simulation,  we  cannot  speak  to 
this  point  directly.  The  reason  is  that  we  are  looking  at  the  long  tail  of  the  distribution  of  tracking 
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Joint  Publication  3-60,  Joint  Targeting,  Washington,  D.C.:  Joint  Chiefs  of  Staff,  April  13,  2007,  defines  a  time- 
sensitive  target  as  a  “joint  force  commander  designated  target  requiring  immediate  response  because  it  is  a  highly 
lucrative,  fleeting  target  of  opportunity  or  it  poses  (or  will  soon  pose)  a  danger  to  friendly  forces.” 
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times:  As  noted  earlier,  when  the  RPA  was  able  to  establish  a  track  for  a  few  minutes,  it  rarely 
lost  track  afterward.  This  suggests  that  an  RPA  that  has  already  tracked  the  target  for  15  to  20 
min.  can  likely  continue  to  do  so  for  quite  some  time — but  how  exactly  long,  we  cannot  say.  In 
reality,  Predators  have  been  known  to  maintain  track  for  much  longer  periods  of  time,  such  that 
even  a  retasking  process  requiring  several  hours  still  led  to  a  successful  kill.  That  such  delays  are 
outside  the  timeframe  of  the  scenario  we  considered  is  a  reminder  that  the  hunter-killer  scenario 
we  examined  is  only  one  of  many. 

Although  we  did  indeed  model  up  to  45  min.  of  operations  beyond  the  initial  arrival  of  the 
target  in  the  kill  zone,  we  do  not  feel  our  FMV  tracking  model  was  sufficiently  robust  to  explore 
that  part  of  the  scenario.54  In  our  case,  we  imposed  a  strict,  continuous  identification 
requirement,  meaning  that  the  sensor  operator  effectively  maintained  eyes  on  target.  Even  so, 
however,  the  cumulative  chance  of  an  unlikely  combination  of  events  occurring  that  could  break 
such  a  track,  even  when  the  vehicle  is  in  view  with  high  image  quality,  will  grow  over  time — and 
we  do  not  have  a  good  model  of  such  “clutter”  for  FMV.  Furthermore,  all  our  assumptions  about 
image  quality  and  enemy  behaviors,  which  are  already  optimistic,  become  increasingly  tenuous 
as  the  time  frame  lengthens.  We  can  say,  however,  that  15  to  20  min.  of  tracking  should  allow, 
after  the  command  and  control  times,  at  least  5  to  10  min.  of  flight  time  for  the  strike  platform  to 
arrive  on  station  (assuming  the  call  for  fires  is  made  promptly),  which,  as  Figure  3.4  shows,  still 
leaves  considerable  range  of  situations  in  which  this  CONOPS  will  succeed.  More-favorable 
assumptions  regarding  command  and  control  times  could  extend  it  further. 

Measures  of  Effectiveness 

We  primarily  examined  two  measures  of  effectiveness:  the  probability  of  identifying  the 
target  and  the  probability  of  successfully  tracking  it  all  the  way  to  the  designated  kill  zone.  (A 
third  measure,  the  average  time  required  to  identify  the  target,  is  discussed  in  the  appendix. 

Given  the  short  time  constraints  of  the  scenario,  this  measure  ultimately  did  not  prove  to  be  of 
sufficient  analytical  value.) 

Because  we  chose  to  calculate  these  two  measures,  the  latter  is  contingent  on  the  former. 

That  is,  in  this  scenario,  a  target  would  not  be  followed  unless  it  could  first  be  identified.  This  is 
a  strong  assumption  that  derives  from  the  constraints  of  the  scenario  we  modeled.  Although  it  is 
perfectly  reasonable  to  track  an  unidentified  object — indeed,  we  implicitly  assumed  that  the  RPA 
sensor  operator  can  “keep  track  of’  the  unidentified  objects  in  his  or  her  field  of  view  by  eye  to 
zoom  in  on  them  one  by  one — in  this  scenario,  there  is  a  short  time  window  in  which  the  target  is 
expected  to  be  visible,  and  the  FOV  of  the  FMV  sensor  is  small  compared  to  the  size  of  the  cued 


54 

The  GMTI  tracking  model,  while  of  course  it  could  always  stand  improvement,  is  not  relevant  in  Case  2  because 
the  RPA  concepts  we  looked  at  that  were  capable  of  carrying  a  GMTI  sensor  were  also  large  enough  to  carry 
weapons. 
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Figure  3.4.  Ranges  from  Flight  Times  for  547  kph  True  Airspeed  (in  minutes) 
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search  area.  As  a  result  of  these  constraints,  once  the  RPA  ceases  its  raster  search  to  follow  a 
target,  it  is  effectively  committed.  The  odds  of  being  able  to  recover  from  such  a  mistake  in  the 
time  available  are  very  small.  Therefore,  in  our  CONOPS,  we  assumed  that  the  pilot  would  not 
divert  to  follow  a  target  unless  the  sensor  operator  could  confirm  to  a  high  degree  of  certainty 
(i.e.,  by  identifying  the  target)  that  it  was  the  right  one.  In  a  different  hunter-killer  scenario,  such 
as  one  in  which  there  is  a  better  initial  cue  for  the  target  or  in  which  there  is  time  to  follow 
multiple  targets,  a  looser  CONOPS  might  well  prove  more  effective. 

Note,  however,  that  all  probabilities  are  reported  in  terms  of  the  whole  number  of  iterations, 
so  if,  in  any  particular  instance,  the  probability  of  identification  equals  the  probability  of 
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maintaining  track,  it  is  correct  to  infer  that  each  target  that  was  successfully  identified  was  also 
successfully  tracked.55 

The  probability  of  merely  detecting  the  target  (as  opposed  to  identifying  it)  was  not 
calculated  separately  and  neither  was  the  probability  of  classification.  These  were  omitted 
because,  under  the  assumption  that  positive  identification  of  the  target  was  the  objective,  we 
selected  the  flight  profile  and  search  pattern  to  provide  sufficient  image  quality  to  make  this 
possible. 


55  We  did  this  to  avoid  reporting  a  probability  of  maintaining  track  that  was  higher  than  that  of  identification,  which 
might  leave  the  incorrect  impression  that  some  targets  that  were  tracked  were  not  identified.  While  it  is  certainly 
possible  to  track  an  unidentified  target,  especially  with  GMTI,  success  in  the  hunter-killer  mission  requires  target 
identification. 
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4.  Analysis  and  Results 


Introduction 

We  assessed  the  operational  effectiveness  of  the  four  different  RPA  concepts  described  in 
Chapter  Two  (MQ-9,  Group  5,  Group  4,  and  Group  3),  employing  a  multiplicity  of  1,  2,  or  3 
platforms  at  a  time,  under  six  different  weather  conditions,  with  two  different  NIIRS 
requirements,  yielding  a  grand  total  of  144  variants.  We  typically  ran  300  to  500  replications  of 
each  variant  for  the  final  production  runs.56  The  full  results  are  tabulated  in  the  appendix.  In  this 
section,  we  summarize  the  results,  pausing  to  discuss  the  more  interesting  outcomes. 


Identifying  and  Tracking  Probabilities 

Clear  Weather 

The  most  favorable  conditions  we  considered  were  broad  daylight,  clear  weather,  and  the 
lower  NIIRS  7.0  requirement  to  identify  the  target.  Under  these  conditions,  as  shown  in 
Figures  4. 1  and  4.2,  all  four  RPA  concepts  performed  very  well  in  terms  of  both  identifying  and 
tracking  the  target.  Groups  3  and  4  fared  a  bit  worse  than  the  others  in  terms  of  probability  of 
tracking,  but  adding  a  second  (or  third)  platform  made  up  for  this  shortfall. 


Figure  4.1.  Probability  of  Identification  in  Broad  Daylight,  Clear  Weather, 
with  NIIRS  7.0  Identification  Requirement 
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56  Increasing  to  1,000  replications  refined  the  average  results  very  little.  Statistically,  the  “error”  due  to  the  smaller 
number  of  replications  was  less  than  1  percent,  but  of  course  the  uncertainty  due  to  the  many  assumptions  outlined 
in  the  previous  two  sections  was  more  significant. 
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Figure  4.2.  Probability  of  Maintaining  Track  in  Broad  Daylight,  Clear  Weather, 
with  NIIRS  7.0  Identification  Requirement 


Prob. 


100% 

90% 

80% 

70% 

60% 

50% 

40% 

30% 

20% 

10% 

0% 


Increased 
effectiveness  from 
additional  RPAs 


Number 
of  RPAs 

■  1 
■  2 

■  3 


MQ-9  Grp.  5  Grp.  4  Grp.  3 


Under  the  same  favorable  environmental  conditions,  increasing  the  identification  requirement 
to  NIIRS  8.0  noticeably  reduces  the  effectiveness  of  all  single  platforms,  notably  the  smaller 
ones  (Group  3  and  Group  4),  but  having  multiple  platforms  compensates  for  most  or  all  of  this 
drop  in  performance.  The  corresponding  charts  are  shown  in  Figures  4.3  and  4.4. 


Figure  4.3.  Probability  of  Identification  in  Broad  Daylight,  Clear  Weather, 
with  NIIRS  8.0  Identification  Requirement 
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Figure  4.4.  Probability  of  Maintaining  Track  in  Broad  Daylight,  Clear  Weather, 
with  NIIRS  8.0  Identification  Requirement 
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Most  hunter-killer  missions  are  currently  conducted  at  night,  however,  due  to  the  obvious 
desire  to  conceal  the  fact  of  surveillance  from  the  target.  So,  we  considered  both  day  and  night 
missions.  Nighttime  identifying  and  tracking  performance  for  NIIRS  7.0  identification 
requirements  are  shown  in  Figures  4.5  and  4.6,  respectively. 


Figure  4.5.  Probability  of  Identification  at  Night,  for  Clear  Weather, 
with  NIIRS  7.0  Identification  Requirement 
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Figure  4.6.  Probability  of  Maintaining  Track  at  Night,  for  Clear  Weather, 
with  NIIRS  7.0  Identification  Requirement 
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The  MTS-B  sensor  package  is  unique  among  the  sensors  considered  in  that,  in  some  of  the 
medium-sized  or  wider  viewing  modes,  the  IR  imaging  quality  is  slightly  superior  to  the  EO 
imaging  quality.  The  ability  to  find  and  identify  any  particular  type  of  target,  however,  still 
depends  on  the  target:  As  noted  earlier,  the  NIIRS  EO  and  IR  scales  are  not  quite  identical  in 
interpretation  or  requirements.  In  this  particular  case,  however,  a  hot  car  engine  at  night  is 
actually  quite  bright  at  IR  wavelengths.  The  MQ-9  and  the  Group  5  concept  therefore  perform 
slightly  better  at  night  than  they  do  during  the  day,  under  certain  conditions. 

Raven  Eye  I  and  Raven  Eye  II  are  general-purpose  surveillance  systems,  however,  and  are 
not  designed  specifically  for  IR  nighttime  searching.  Unlike  the  MTS-B,  the  IR  image  quality  for 
these  sensors  is  noticeably  poorer  than  the  corresponding  EO  image.  The  resulting  flagging 
performance  at  night  in  the  hunter-killer  scenario  we  considered  is  evident.  Even  multiple 
Group  3  and  Group  4  platforms  cannot  compensate  for  the  loss  of  effectiveness  because  of  the 
less  capable  IR  sensors  they  are  constrained  to  carry. 

Moreover,  the  drop  in  performance  for  Groups  3  and  4  is  due  primarily  to  the  need  to  fly  at 
much  lower  altitudes  to  achieve  the  required  image  quality  to  identify  the  target.  For  this  reason, 
it  takes  longer  for  them  to  search  the  area,  so  they  have  a  greater  likelihood  of  missing  the  target 
entirely.  The  tracking  performance  is  affected  similarly  because,  at  lower  altitudes,  the  field  of 
view  is  smaller,  and  it  is  easier  to  lose  the  target  when  it  makes  an  unexpected  turn.  Raising  the 
altitude  did  not  prove  helpful  because,  in  the  event  that  the  track  is  lost,  the  RPA  must  reacquire 
and  reidentify. 

The  corresponding  identification  and  tracking  probabilities  would  be  similarly  pronounced 
for  NIIRS  8.0  identification  requirements  (not  shown).57  As  in  the  daylight  case,  much  of  the 
benefit  of  having  additional  platforms  derives  from  the  second  platform;  there  is  only  marginal 


57 


For  this  data,  see  the  appendix. 
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improvement  with  the  third.  It  is  also  worth  noting  that  difficulties  in  tracking  were  almost 
always  evident  from  the  beginning  of  the  replication,  when  the  RPA  attempted  to  follow  the 
target  out  of  the  urban  canyon  in  the  early  stages  of  the  scenario.  The  upshot  of  this  is  that,  if  the 
RPA  was  in  fact  able  to  achieve  a  stable  track  on  the  target  for  the  first  few  minutes,  it  was 
usually  able  to  continue  to  do  so  throughout. 

Fog 

We  considered  a  moderate  fog  with  visibility  that  varied  considerably  with  viewing  angle 
and  altitude.  Depending  on  the  circumstances,  the  transmittance  ratio  for  visible  wavelengths 
could  vary  from  0.0  (complete  opacity)  to  about  0.5.  The  fog  was  more  penetrable  by  IR 
wavelengths,  which  allowed  approximately  0.9  transmittance  for  range  of  viewing  angles  (but 
still  went  to  near-zero  visibility  for  the  most  shallow  viewing  conditions). 

The  most  significant  effect  of  fog  was  on  daytime  EO  sensors,  with  the  more  challenging 
NIIRS  8.0  requirement.  The  results  for  daytime  identification  and  tracking  probabilities  are 
shown  in  Figures  4.7  and  4.8,  respectively.  The  corresponding  results  for  nighttime  follow  the 
same  lines  as  those  without  fog,  because  IR  sensors  can  penetrate  fog  more  effectively  than  EO 
sensors  can. 


Figure  4.7.  Probability  of  Identification  for  Daytime  Fog, 
NIIRS  8.0  Requirement 
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Figure  4.8.  Probability  of  Maintaining  Track  for  Daytime  Fog, 
NIIRS  8.0  Requirement 
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Cloud  Cover 

The  performance  of  the  RPAs  under  conditions  of  cloud  cover  is  quite  different  from  under 
conditions  of  fog.  We  took  a  partly  cloudy  sky  of  40-percent  average  coverage,  at  a  cloud  ceiling 
of  about  3,000  ft.  above  the  terrain.58  Two  examples  of  the  terrain  with  cloud  coverage,  at 
different  times,  are  shown  in  Figure  4.9.  The  cloud  cover  pattern  changed  slowly  throughout  the 
simulation. 

The  probabilities  of  finding  and  identifying  the  target  and  of  maintaining  track  through  the 
urban  canyon  to  the  target’s  destination  are  shown  in  Figures  4.10  and  4.11.  Because  the  MQ-9 
and  Group  5  concepts  were  equipped  with  the  Lynx  GMTI/SAR  radar  system  (in  addition  to  the 
FMV  modality),  they  were  able  to  penetrate  the  cloud  cover  to  maintain  track  (although  not  for 
identification  because  that  still  required  a  solid  image  in  EO  or  IR).  Given  that  the  conditions 
were  not  fully  cloudy,  it  was  advantageous  for  these  two  larger  RPAs  to  continue  to  fly  above  the 
clouds,  as  though  the  clouds  were  absent.  However,  the  Group  3  and  Group  4  concepts,  which 
did  not  have  radar  capabilities,  needed  to  fly  below  the  clouds,  otherwise  they  would  be 
completely  ineffective.  These  choices  had  a  significant  effect  on  the  results.  In  the  cases  shown 
here,  the  Group  3  concepts  outperfonn  Group  4  somewhat  because  of  improved  maneuverability 
at  slower  speeds;  however,  at  night  (see  the  appendix),  this  relative  performance  is  reversed 
because  the  improved  IR  capabilities  of  the  heavier  sensor  package  on  the  Group  4  aircraft 
become  the  controlling  factor. 


58 

We  used  the  center  of  the  urban  area  as  the  reference  point.  As  the  target  headed  toward  more-mountainous  rural 
areas,  the  effective  height  of  the  cloud  ceiling  changed  as  the  road  rose.  Generally  speaking,  the  effective  height  of 
the  cloud  ceiling  was  lower  in  rural  areas. 
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Figure  4.9.  Cloud  Cover  over  the  Region  of  Interest  at  Different  Times 


NOTE:  Black  is  obscure;  pale  blue  is  clear  sky.  Terrain  is  also  shown. 

Figure  4.10.  Probability  of  Identification  in  Daytime,  with  Clouds, 
NIIRS  7.0  Identification  Requirement 
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Figure  4.11.  Probability  of  Maintaining  Track  on  Target  in  Daytime,  with  Clouds 
NIIRS  7.0  Identification  Requirement 
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One  unexpected  result  is  that  the  MQ-9  was  the  best  single  platform  in  terms  of  maintaining 
track  and  was  nearly  tied  for  best  as  a  single  platform  in  terms  of  identifying  the  target.  One 
interpretation  of  these  results,  which  we  discuss  further  in  Chapter  Four,  is  that  the  MQ-9  was 
developed  for  this  particular  kind  of  mission  in  Iraq  and  Afghanistan.  However,  adding 
additional  Group  3  and  Group  4  platforms  significantly  increased  effectiveness,  while  adding 
additional  MQ-9  platforms  gained  little.  This  is  because  these  two  smaller  groups  fly  below  the 
clouds,  and  therefore  it  is  the  altitude  that  restricts  them,  rather  than  the  blockage  of  LOS. 

The  situation  becomes  clearer  if  we  look  at  the  nighttime  figures.  As  noted  earlier,  many 
missions  in  Iraq  and  Afghanistan  were  night  missions,  and  the  MTS-B  sensor  has  unusually 
powerful  IR  capabilities.  Figures  4.12  and  4.13  show  the  probabilities  of  identifying  and 
maintaining  track,  respectively.  For  a  single  platfonn,  the  MQ-9  is  best  in  both  measures  but 
ends  up  in  the  middle  of  the  pack  with  others  when  multiple  platforms  are  considered. 

One  feature  of  interest  is  that  the  Group  5  platfonn  does  poorest  in  tracking  in  daytime  and 
nighttime  cases.  This  is  because  the  Group  5  platform  we  modeled,  a  jet,  has  a  minimum  speed 
that  is  much  faster  than  that  of  the  vehicle  it  is  attempting  to  track  (see  Table  1.1).  This  mismatch 
in  velocities  makes  tracking  difficult.  As  a  result,  the  Group  5  platfonn  loses  the  target  more 
frequently  behind  clouds  and  is  forced  into  unfavorable  geometries  more  often  as  it  tries  to  keep 
the  target  in  view.  It  is  also  not  clear  that  it  can  react  to  changes  in  target  movement  as 
effectively. 
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Figure  4.12.  Probability  of  Identification  at  Night,  with  Clouds 
NIIRS  7.0  Identification  Requirement 
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Figure  4.13.  Probability  of  Maintaining  Track  on  Target  at  Night,  with  Clouds, 
NIIRS  7.0  Identification  Requirement 
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Summary  Figures 

The  following  stoplight  charts  summarize  the  operational  effectiveness  of  all  cases 
considered  in  this  analysis.  We  show  here  the  results  for  one  platform  and  for  three  platforms. 
(The  appendix  offers  the  intermediate  results  for  two  platforms.)  Results  for  the  probability  of 
identification  are  shown  in  Figure  4.14. 

A  similar  chart  for  maintaining  track  is  shown  in  Figure  4.15.  In  both  charts,  the  inability  of 
a  single  Group  3  or  Group  4  RPA  concept  to  fulfill  mission  requirements  is  evident,  but  the 
remarkable  improvement  due  to  employing  multiple  platforms  in  a  cooperative  fashion  is  evident 
as  well. 
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Night  Day  Night 


Figure  4.14.  Summary  of  Probability  of  Identification,  All  Cases 
for  One  or  Three  RPAs 
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Figure  4.15.  Summary  of  Probability  of  Maintaining  Track,  All  Cases, 
for  One  or  Three  RPAs 
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5.  Conclusion 


Findings 

Analysis  of  the  modeling  results  leads  us  to  four  broad  findings. 

First,  there  is  no  silver  bullet.  Even  in  this  one  particular  type  of  one  particular  hunter-killer 
mission,  no  single  RPA  concept  performed  well  on  all  measures  under  all  environmental 
conditions.  In  fact,  there  is  at  least  one  case  in  which  each  of  the  four  RPA  concepts  perfonned 
better  than  the  others.  For  example,  with  multiple  aircraft,  the  low-flying  Group  3  perfonned 
better  at  tracking  and  identification  under  cloud  cover  during  the  day,  but  Group  4  was  somewhat 
better  with  cloud  cover  at  night.  Meanwhile,  Group  5  generally  perfonned  best  in  terms  of 
probability  of  identification,  but  the  MQ-9  edged  it  out  in  some  situations  when  it  comes  to 
probability  of  maintaining  track.  Note,  however,  that  platforms  with  inferior  sensors  must  fly  at 
lower  altitudes  to  achieve  the  required  resolution,  which  clearly  would  entail  greater 
survivability  concerns  in  any  threat  environment  other  than  the  permissive  environment  we 
considered. 

Second,  numbers  can  compensate  for  capability.  Despite  their  inferior-resolution  sensor 
suites,  multiple  smaller  RPAs  (Groups  3  and  4)  equaled  or  exceeded  the  performance  of  single 
larger  RPAs  in  many  environmental  conditions  (MQ-9  and  Group  5),  especially  in  the  case  of 
cloud  cover.59  The  only  clear  exception,  however,  is  at  night,  when  the  superior  MTS-B  IR 
sensor  demonstrated  a  clear  advantage.  With  sufficient  numbers,  all  platforms  are  able  to  detect, 
identify,  and  track  the  target  for  at  least  15  min. — long  enough  for  to  achieve  a  kill  by  another 
fighter  within  50  nmi,  including  the  time  required  to  complete  all  command  and  control 
processes  (on  average).  Multiple  platforms  are  also  less  likely  to  lose  targets  in  the  urban  canyon, 
thanks  to  multiple  viewpoints.  Smaller  RPAs  may  also  be  able  to  augment  capabilities  against 
smart,  sophisticated  adversaries. 

Third,  the  MQ-9  Reaper  holds  up  well  against  the  other  RPA  design  concepts  we  modeled  in 
this  scenario.  The  MQ-9  scored  comparably  to  other  options  under  most  environmental 
conditions.  It  rarely  outperformed  all  other  platforms  but  betrayed  no  Achilles’  heel:  The  MQ-9 
was  never  dramatically  outperformed  and  never  fared  worst  on  any  measure.  It  was  the  only  one 
of  the  four  RPA  concepts  considered  that  never  demonstrated  a  probability  of  identification  or 
probability  of  maintaining  track  of  less  than  50  percent  as  a  single  platform.  (Multiple  platforms 
increase  the  scores.)  The  ability  to  fly  significantly  slower,  along  with  a  tighter  turn  radius, 


59 

We  used  an  obscuration  of  40  percent.  Although  we  did  not  simulate  more  opaque  cloud  cover  conditions,  we 
believe  it  likely  that  the  Group  3  and  Group  4  platforms  would  also  perform  well  in  those  cases. 
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allows  MQ-9  to  track  ground  vehicles  well;  the  MTS-B  and  Lynx  combination  provides  night¬ 
time,  fog,  and  partial  cloud-cover  capabilities.60 

Finally,  improving  MQ-9  sensor  capabilities  may  be  a  cost-effective  option.  Although  we  did 
not  perform  a  cost-effectiveness  analysis,  it  appears  that  upgrading  the  existing  MQ-9  platform 
in  this  way  may  be  a  preferred  way,  given  budget  constraints,  to  achieve  moderately  enhanced 
operational  effectiveness  in  similar  scenarios. 

One  modest  enhancement  would  be  to  make  the  MTS-B  sensor  package  more  flexible  by 
adding  variable  (as  opposed  to  fixed)  levels  of  magnification.  Neither  the  EO  nor  IR  video 
cameras  on  the  MTS-B  are  capable  of  continuous  zoom — the  MTS-B  allows  only  fixed  levels  of 
magnification — and  the  platform  was  thus  often  unable  to  take  advantage  of  a  “sweet  spot”  for 
this  mission  in  terms  of  altitude  and  flight  pattern.  This  would  have  improved  the  CONOPS  and 
altered  the  flight  profiles. 

A  more-significant  enhancement  would  be  to  improve  the  effective  image  quality  of  MTS-B 
video  by  +1.0  NIIRS.61  This  would  not  merely  improve  the  picture  on  the  screen  but,  more 
important,  would  pennit  greater  operational  flexibility.  An  improvement  of  +  1.0  NIIRS  would 
allow  the  Reaper  to  identify  the  target  at  nearly  twice  the  altitude  under  many  conditions.  This 
would  allow  the  platform  not  only  to  search  more  effectively  but  to  track  more  effectively 
because  it  would  not  be  as  vulnerable  to  losing  a  target  that  changed  its  route  unexpectedly,  and 
the  higher  vantage  point  would  afford  greater  visibility  into  the  urban  canyon.  It  would  also 
improve  survivability.  Along  the  same  lines,  an  improved  WAMI  sensor,  with  enhanced  NIIRS 
image  quality,  could  prove  very  useful  as  well,  provided  that  PED  resources  were  also  available 
to  handle  the  higher  data  rate. 

Areas  for  Further  Analysis 

One  excursion  we  did  not  consider  was  pairing  different  RPA  classes  together.  In  particular, 
we  did  not  consider  the  advantages  of  having  one  smaller  RPA  fly  below  the  clouds  while  a 
larger  one  flew  above.  Once  the  CONOPS  for  such  a  mission  are  understood,  this  would  be  an 
important  case,  looking  to  see  if  synergies  between  the  two  different  classes  can  be 
demonstrated,  at  least  in  this  relatively  simple  scenario. 

We  also  looked  at  only  one  particular  version  of  cloud  cover,  yet  the  ceiling  and  opacity  can 
vary  considerably,  and  we  also  looked  only  at  one  instance  of  fog.  While  the  results  suggest  that 
increasing  the  intensity  of  fog  would  produce  fairly  straightforward  extrapolation,  it  is  not  clear 
how  the  balance  between  flying  above  and  below  the  clouds  would  work  for  more-challenging 
cloud  cover  situations.  In  very  opaque  cases  with  higher  cloud  cover,  it  may  be  sensible  for  the 


60  It  does  not  provide  all-weather  capabilities.  As  was  pointed  out  to  us  by  more  than  one  Air  Force  officer,  there  is 
always  a  reasonable  combination  of  weather  that  can  thwart  airborne  ISR  operations. 

61  Greater  improvements  in  NIIRS  are  of  course  more  desirable,  but  even  a  single  level  of  improvement  would  make 
a  big  difference. 
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MQ-9  and  even  Group  5  to  fly  below  the  cloud  ceiling,  but  we  did  not  demonstrate  that  in  this 
report. 

More  important,  we  employed  a  simple  CONOPS  for  the  enemy  that  was  neither  adaptive 
nor  reactive.  Other  than  occasionally  stopping  to  attempt  to  elude  pursuit,  the  target  in  this 
scenario  did  not  use  camouflage  or  concealment.  We  also  did  not  consider  the  possibility  that  the 
targeted  individual  could  hear  or  see  the  RPA  tracking  him  or  her  and  respond  by  hiding  or 
changing  vehicles.62  Modeling  a  more-intelligent  or  more-cautious  enemy  would  be  a  valuable 
addition.  A  more-robust  tracking  model  would  also  be  necessary  in  that  case. 

We  also  did  not  model  the  Gorgon  Stare  or  future  WAMI  sensors,  largely  because  of  a  lack 
of  a  satisfactory  model  for  the  PED  resources  required.  To  make  a  fair  comparison  between 
WAMI  sensors  and  the  others  we  considered  would  have  required  addressing  these  PED  issues, 
and  we  chose  not  to  focus  on  that  area.  As  WAMI  sensors  are  now  on  the  battlefield,  this  is  an 
important  area  for  future  research. 

We  also  did  not  consider  upgraded  versions  of  the  sensors  we  did  model.  To  the  extent  that 
such  upgrades  would  merely  increase  the  image  quality  at  a  given  altitude,  the  performance  can 
be  extrapolated  easily.  However,  in  some  cases,  dramatically  improved  sensor  performance 
might  permit  the  RPA  to  take  advantage  of  different  CONOPS  to  achieve  even  greater 
operational  effectiveness.  Enhancing  the  modeling  effort  to  account  for  more-capable  enemies, 
and  more-capable  assets,  would  be  another  important  future  contribution  to  the  analysis  of 
hunter-killer  and  similar  missions. 


6“  The  success  of  counterterrorism  operations  with  the  MQ-1  and  MQ-9  in  Iraq  and  Afghanistan  demonstrates  that 
they  are  generally  able  to  fly  such  that  they  are  difficult  to  see  or  hear,  especially  in  an  urban  environment — and 
previous  methodology  work  on  aural  detection  is  not  inconsistent  with  this — but  we  have  no  such  evidence  for  either 
the  much-lower-flying  Group  3  and  Group  4  in  this  type  of  mission. 
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Appendix.  Full  Results 


Before  presenting  the  complete  numerical  modeling  results,  we  will  introduce  and  describe  a 
third  measure  of  effectiveness — the  time  to  identification — which  is  included  here  alongside  the 
two  measures  reported  in  the  body  of  the  report.  Although  this  measure  was  excluded  from  the 
write-up  because  it  ultimately  lacked  sufficient  analytical  value,  we  include  it  here  in  the 
interests  of  completeness  and  because  it  may  offer  insights  into  the  modeling  approach. 

Time  to  identification  is  defined  as  the  total  elapsed  time  between  the  emergence  of  the  target 
from  its  (random)  hiding  place  and  its  initial  identification  by  the  RPA  sensor  operator.  Note  that 
only  the  first  successful  identification  is  included.  If  the  target  is  lost  and  must  be  reacquired,  the 
measure  is  not  reset. 

Because  this  metric  is  contingent  on  the  probability  of  identification,  this  creates  some 
unwelcome  artifacts.  In  particular,  the  average  time  to  identification  occasionally  appears  to  be 
slightly  worse  (higher)  when  additional  platforms  are  added.  This  is  because  they  are  able  to 
locate  the  target  vehicle  under  more  challenging  conditions — and  when  these  harder-won 
successes  are  added  to  the  sample,  it  can  skew  the  average  time  to  identification  upward. 

The  results  for  the  best  viewing  conditions  (daytime  and  clear  weather)  and  least  stringent 
identification  requirements  (NIIRS  7.0)  are  shown  in  Figure  A.l.  In  this  situation,  nearly  all 
platforms  saw  all  targets,  so  the  results  are  straightforward:  The  platforms  that  moved  the  fastest 
(Group  5)  were  the  first  to  identify,  and  adding  extra  platforms  reduced  the  time  to  identification 
similarly. 


Figure  A.l.  Average  Time  to  Identification  in  Broad  Daylight, 
Clear  Weather,  with  NIIRS  7.0  Identification  Requirement 
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Adding  fog,  changing  to  nighttime  operations,  or  raising  the  NIIRS  identification  threshold  to 
8.0  does  not  alter  this  underlying  pattern.  In  most  cases,  the  pattern  is  instead  exaggerated.  As  an 
illustration,  Figure  A. 2  shows  the  results  of  including  all  three  changes  at  once. 

The  results  for  the  Group  3  aircraft  illustrate  the  artifact  mentioned  previously.  The 
performance  of  a  single  Group  3  aircraft  was  so  poor  (29  percent)  that  the  only  targets  it  could 
identify  were  those  that  were  easiest  to  find.  Adding  a  second  or  third  RPA  immediately  bumped 
the  identification  rate  over  70  percent,  and  the  sample  then  included  more  of  the  targets  that 
innately  took  longer  to  identify. 

Cloud  cover  noticeably  shifts  the  overall  pattern,  however.  The  results  for  cloud  cover, 
daylight,  and  NIIRS  8.0  identification  threshold  are  shown  in  Figure  A. 3.  In  this  variant, 

Groups  3  and  4  can  achieve  faster  identification  times  because  they  fly  beneath  the  clouds;  in 
contrast,  the  MQ-9  and  Group  5  platform  must  wait  for  the  target  to  pass  through  a  break  in  the 
clouds  to  positively  identify  it  with  FMV.  Although,  in  principle,  the  MQ-9  and  Group  5  can  fly 
beneath  the  clouds,  the  cloud  ceiling  we  modeled  was  very  low  (3,000  ft.),  and  this  is  generally 
considered  impractical.  Operating  one  of  these  large  RPAs  so  close  to  the  ground  also  would 
have  violated  our  assumption  that  the  enemy  could  not  see  or  hear  it. 

Figure  A.2.  Average  Time  to  Identification  at  Night,  with  Fog, 
with  NIIRS  8.0  Identification  Requirement 
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Figure  A.3.  Average  Time  to  Identification  in  Broad  Daylight,  with  Clouds, 
with  NIIRS  8.0  Identification  Requirement 
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Further  results  for  this  metric  may  be  found  in  Table  A.l,  which  summarizes  the  outcomes  of 
all  three  measures  of  effectiveness  for  all  cases.  In  this  table,  “Found”  and  “Tracked”  refer  to  the 
probability  of  identification  and  the  probability  of  following  the  target  vehicle  all  the  way  to  the 
kill  zone,  respectively.  “Avg.  Time  to  ID”  refers  to  the  third  metric  discussed  above.  A  fourth 
statistic,  “Avg.  Track  Duration,”  refers  only  to  the  lengths  of  tracks  that  were  not  ultimately 
successful.  In  some  cases,  no  figure  is  provided,  because  every  identified  target  was  also 
successfully  tracked  all  the  way  to  the  kill  zone.  Because  this  statistic  depends  so  strongly  on  the 
arbitrary  starting  and  ending  points  for  the  particular  replications,  it  has  very  limited  analytic 
value  and  is  essentially  useful  only  for  diagnostic  purposes.  We  included  it  here  for  reference 
only. 
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Table  A.1.  Numerical  Modeling  Results 


Time 

Weather 

ID  Req. 

Platform 

Multiplicity 

Found 

(percent) 

Tracked 

(percent) 

Avg.  Track 
Duration 
(min.) 

Avg.  Time 
to  ID 
(min.) 

Day 

Clear 

NIIRS  7.0 

MQ-9 

1 

96 

96 

24.71 

6.39 

Day 

Clear 

NIIRS  7.0 

MQ-9 

2 

99 

99 

20.46 

4.58 

Day 

Clear 

NIIRS  7.0 

MQ-9 

3 

99 

99 

27.85 

4.05 

Day 

Clear 

NIIRS  7.0 

Group  5 

1 

98 

97 

25.70 

4.35 

Day 

Clear 

NIIRS  7.0 

Group  5 

2 

98 

98 

23.56 

2.99 

Day 

Clear 

NIIRS  7.0 

Group  5 

3 

98 

98 

34.45 

2.79 

Day 

Clear 

NIIRS  7.0 

Group  4 

1 

98 

75 

18.94 

6.44 

Day 

Clear 

NIIRS  7.0 

Group  4 

2 

99 

92 

18.55 

4.98 

Day 

Clear 

NIIRS  7.0 

Group  4 

3 

100 

98 

23.03 

3.76 

Day 

Clear 

NIIRS  7.0 

Group  3 

1 

94 

76 

21.50 

8.12 

Day 

Clear 

NIIRS  7.0 

Group  3 

2 

99 

93 

16.52 

5.92 

Day 

Clear 

NIIRS  7.0 

Group  3 

3 

100 

99 

13.15 

4.27 

Day 

Clear 

NIIRS  8.0 

MQ-9 

1 

87 

87 

37.15 

8.78 

Day 

Clear 

NIIRS  8.0 

MQ-9 

2 

99 

99 

— 

6.43 

Day 

Clear 

NIIRS  8.0 

MQ-9 

3 

99 

99 

21.13 

4.68 

Day 

Clear 

NIIRS  8.0 

Group  5 

1 

96 

96 

22.98 

6.67 

Day 

Clear 

NIIRS  8.0 

Group  5 

2 

99 

97 

31.47 

4.23 

Day 

Clear 

NIIRS  8.0 

Group  5 

3 

99 

99 

— 

4.65 

Day 

Clear 

NIIRS  8.0 

Group  4 

1 

82 

47 

22.91 

8.99 

Day 

Clear 

NIIRS  8.0 

Group  4 

2 

96 

83 

20.91 

6.88 

Day 

Clear 

NIIRS  8.0 

Group  4 

3 

100 

94 

21.57 

5.20 

Day 

Clear 

NIIRS  8.0 

Group  3 

1 

70 

49 

25.94 

9.57 

Day 

Clear 

NIIRS  8.0 

Group  3 

2 

91 

86 

24.69 

8.08 

Day 

Clear 

NIIRS  8.0 

Group  3 

3 

96 

95 

28.11 

7.09 

Day 

Clouds 

NIIRS  7.0 

MQ-9 

1 

75 

71 

24.50 

8.54 

Day 

Clouds 

NIIRS  7.0 

MQ-9 

2 

78 

76 

32.63 

10.02 

Day 

Clouds 

NIIRS  7.0 

MQ-9 

3 

79 

76 

36.83 

10.25 

Day 

Clouds 

NIIRS  7.0 

Group  5 

1 

67 

40 

20.30 

5.35 

Day 

Clouds 

NIIRS  7.0 

Group  5 

2 

74 

56 

21.91 

6.02 

Day 

Clouds 

NIIRS  7.0 

Group  5 

3 

77 

60 

24.53 

4.71 

Day 

Clouds 

NIIRS  7.0 

Group  4 

1 

60 

38 

23.66 

9.03 

Day 

Clouds 

NIIRS  7.0 

Group  4 

2 

80 

76 

32.04 

7.57 

Day 

Clouds 

NIIRS  7.0 

Group  4 

3 

91 

88 

29.18 

6.81 

Day 

Clouds 

NIIRS  7.0 

Group  3 

1 

78 

58 

26.20 

9.40 

Day 

Clouds 

NIIRS  7.0 

Group  3 

2 

92 

85 

27.33 

6.95 

Day 

Clouds 

NIIRS  7.0 

Group  3 

3 

99 

98 

31.48 

5.50 
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Table  A.1 — Continued 


Time 

Weather 

ID  Req. 

Platform 

Multiplicity 

Found 

(percent) 

Tracked 

(percent) 

Avg.  Track 
Duration 
(min.) 

Avg.  Time 
to  ID 
(min.) 

Day 

Clear 

NIIRS  8.0 

MQ-9 

1 

62 

59 

28.87 

9.97 

Day 

Clear 

NIIRS  8.0 

MQ-9 

2 

76 

75 

30.76 

10.20 

Day 

Clear 

NIIRS  8.0 

MQ-9 

3 

79 

77 

35.39 

7.82 

Day 

Clouds 

NIIRS  8.0 

Group  5 

1 

72 

50 

18.35 

8.96 

Day 

Clouds 

NIIRS  8.0 

Group  5 

2 

77 

60 

22.04 

8.37 

Day 

Clouds 

NIIRS  8.0 

Group  5 

3 

77 

61 

25.03 

8.64 

Day 

Clouds 

NIIRS  8.0 

Group  4 

1 

57 

36 

22.28 

8.74 

Day 

Clouds 

NIIRS  8.0 

Group  4 

2 

81 

74 

31.41 

7.90 

Day 

Clouds 

NIIRS  8.0 

Group  4 

3 

94 

92 

27.86 

7.26 

Day 

Clouds 

NIIRS  8.0 

Group  3 

1 

70 

47 

22.57 

9.75 

Day 

Clouds 

NIIRS  8.0 

Group  3 

2 

91 

82 

14.78 

7.70 

Day 

Clouds 

NIIRS  8.0 

Group  3 

3 

95 

95 

60.33 

6.06 

Day 

Fog 

NIIRS  7.0 

MQ-9 

1 

94 

94 

29.46 

8.20 

Day 

Fog 

NIIRS  7.0 

MQ-9 

2 

100 

100 

— 

5.34 

Day 

Fog 

NIIRS  7.0 

MQ-9 

3 

100 

99 

33.00 

3.99 

Day 

Fog 

NIIRS  7.0 

Group  5 

1 

99 

99 

22.57 

4.75 

Day 

Fog 

NIIRS  7.0 

Group  5 

2 

100 

100 

22.66 

4.59 

Day 

Fog 

NIIRS  7.0 

Group  5 

3 

100 

99 

21.07 

3.27 

Day 

Fog 

NIIRS  7.0 

Group  4 

1 

96 

57 

22.97 

7.39 

Day 

Fog 

NIIRS  7.0 

Group  4 

2 

97 

85 

15.52 

7.60 

Day 

Fog 

NIIRS  7.0 

Group  4 

3 

100 

99 

18.03 

4.16 

Day 

Fog 

NIIRS  7.0 

Group  3 

1 

80 

49 

20.55 

9.52 

Day 

Fog 

NIIRS  7.0 

Group  3 

2 

98 

84 

19.71 

6.49 

Day 

Fog 

NIIRS  7.0 

Group  3 

3 

100 

98 

17.34 

5.28 

Day 

Fog 

NIIRS  8.0 

MQ-9 

1 

76 

76 

29.57 

7.65 

Day 

Fog 

NIIRS  8.0 

MQ-9 

2 

87 

87 

67.37 

7.12 

Day 

Fog 

NIIRS  8.0 

MQ-9 

3 

97 

97 

38.86 

5.82 

Day 

Fog 

NIIRS  8.0 

Group  5 

1 

90 

89 

36.73 

6.32 

Day 

Fog 

NIIRS  8.0 

Group  5 

2 

96 

96 

73.96 

4.90 

Day 

Fog 

NIIRS  8.0 

Group  5 

3 

98 

98 

— 

4.16 

Day 

Fog 

NIIRS  8.0 

Group  4 

1 

75 

44 

20.86 

9.92 

Day 

Fog 

NIIRS  8.0 

Group  4 

2 

93 

82 

28.81 

7.28 

Day 

Fog 

NIIRS  8.0 

Group  4 

3 

100 

93 

11.27 

6.43 

Day 

Fog 

NIIRS  8.0 

Group  3 

1 

44 

26 

26.15 

10.44 

Day 

Fog 

NIIRS  8.0 

Group  3 

2 

73 

66 

24.51 

7.72 

Day 

Fog 

NIIRS  8.0 

Group  3 

3 

84 

83 

27.43 

6.90 
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Table  A.1 — Continued 


Time 

Weather 

ID  Req. 

Platform 

Multiplicity 

Found 

(percent) 

Tracked 

(percent) 

Avg.  Track 
Duration 
(min.) 

Avg.  Time 
to  ID 
(min.) 

Night 

Clear 

NIIRS  7.0 

MQ-9 

1 

97 

96 

34.18 

7.83 

Night 

Clear 

NIIRS  7.0 

MQ-9 

2 

100 

100 

26.64 

5.59 

Night 

Clear 

NIIRS  7.0 

MQ-9 

3 

100 

100 

26.74 

4.09 

Night 

Clear 

NIIRS  7.0 

Group  5 

1 

100 

99 

25.44 

4.97 

Night 

Clear 

NIIRS  7.0 

Group  5 

2 

100 

100 

19.77 

4.44 

Night 

Clear 

NIIRS  7.0 

Group  5 

3 

100 

100 

29.18 

3.11 

Night 

Clear 

NIIRS  7.0 

Group  4 

1 

57 

35 

27.94 

9.65 

Night 

Clear 

NIIRS  7.0 

Group  4 

2 

88 

86 

28.47 

8.17 

Night 

Clear 

NIIRS  7.0 

Group  4 

3 

93 

92 

30.90 

6.30 

Night 

Clear 

NIIRS  7.0 

Group  3 

1 

32 

16 

26.45 

5.96 

Night 

Clear 

NIIRS  7.0 

Group  3 

2 

61 

54 

35.71 

7.83 

Night 

Clear 

NIIRS  7.0 

Group  3 

3 

74 

72 

26.10 

6.99 

Night 

Clear 

NIIRS  8.0 

MQ-9 

1 

92 

91 

42.22 

7.29 

Night 

Clear 

NIIRS  8.0 

MQ-9 

2 

98 

98 

36.47 

5.84 

Night 

Clear 

NIIRS  8.0 

MQ-9 

3 

99 

99 

34.88 

5.27 

Night 

Clear 

NIIRS  8.0 

Group  5 

1 

97 

97 

22.63 

6.11 

Night 

Clear 

NIIRS  8.0 

Group  5 

2 

100 

98 

21.86 

4.73 

Night 

Clear 

NIIRS  8.0 

Group  5 

3 

99 

99 

— 

5.43 

Night 

Clear 

NIIRS  8.0 

Group  4 

1 

60 

40 

26.50 

10.25 

Night 

Clear 

NIIRS  8.0 

Group  4 

2 

86 

81 

26.79 

8.38 

Night 

Clear 

NIIRS  8.0 

Group  4 

3 

97 

96 

25.92 

6.69 

Night 

Clear 

NIIRS  8.0 

Group  3 

1 

36 

18 

28.18 

8.27 

Night 

Clear 

NIIRS  8.0 

Group  3 

2 

69 

64 

38.96 

9.10 

Night 

Clear 

NIIRS  8.0 

Group  3 

3 

82 

81 

31.79 

7.96 

Night 

Clouds 

NIIRS  7.0 

MQ-9 

1 

76 

74 

26.18 

11.63 

Night 

Clouds 

NIIRS  7.0 

MQ-9 

2 

82 

81 

32.30 

10.18 

Night 

Clouds 

NIIRS  7.0 

MQ-9 

3 

81 

80 

35.40 

9.31 

Night 

Clouds 

NIIRS  7.0 

Group  5 

1 

73 

58 

21.21 

8.60 

Night 

Clouds 

NIIRS  7.0 

Group  5 

2 

86 

68 

23.04 

8.43 

Night 

Clouds 

NIIRS  7.0 

Group  5 

3 

89 

71 

24.36 

9.11 

Night 

Clouds 

NIIRS  7.0 

Group  4 

1 

57 

35 

27.94 

9.65 

Night 

Clouds 

NIIRS  7.0 

Group  4 

2 

88 

86 

28.47 

8.17 

Night 

Clouds 

NIIRS  7.0 

Group  4 

3 

92 

91 

35.93 

6.27 

Night 

Clouds 

NIIRS  7.0 

Group  3 

1 

32 

16 

26.45 

5.96 

Night 

Clouds 

NIIRS  7.0 

Group  3 

2 

61 

54 

35.71 

7.83 

Night 

Clouds 

NIIRS  7.0 

Group  3 

3 

74 

72 

26.10 

6.99 
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Table  A.1 — Continued 


Time 

Weather 

ID  Req. 

Platform 

Multiplicity 

Found 

(percent) 

Tracked 

(percent) 

Avg.  Track 
Duration 
(min.) 

Avg.  Time 
to  ID 
(min.) 

Night 

Clouds 

NIIRS  8.0 

MQ-9 

1 

60 

58 

31.23 

9.35 

Night 

Clouds 

NIIRS  8.0 

MQ-9 

2 

73 

71 

33.62 

9.46 

Night 

Clouds 

NIIRS  8.0 

MQ-9 

3 

81 

79 

34.32 

8.01 

Night 

Clouds 

NIIRS  8.0 

Group  5 

1 

73 

54 

18.42 

8.87 

Night 

Clouds 

NIIRS  8.0 

Group  5 

2 

84 

68 

24.32 

8.62 

Night 

Clouds 

NIIRS  8.0 

Group  5 

3 

88 

75 

23.88 

8.06 

Night 

Clouds 

NIIRS  8.0 

Group  4 

1 

59 

32 

27.84 

10.21 

Night 

Clouds 

NIIRS  8.0 

Group  4 

2 

86 

81 

26.79 

8.38 

Night 

Clouds 

NIIRS  8.0 

Group  4 

3 

93 

92 

20.37 

6.39 

Night 

Clouds 

NIIRS  8.0 

Group  3 

1 

26 

13 

20.85 

6.92 

Night 

Clouds 

NIIRS  8.0 

Group  3 

2 

67 

60 

37.57 

9.91 

Night 

Clouds 

NIIRS  8.0 

Group  3 

3 

79 

78 

43.72 

7.86 

Night 

Fog 

NIIRS  7.0 

MQ-9 

1 

94 

93 

31.19 

7.95 

Night 

Fog 

NIIRS  7.0 

MQ-9 

2 

99 

99 

22.00 

5.75 

Night 

Fog 

NIIRS  7.0 

MQ-9 

3 

100 

99 

29.45 

4.17 

Night 

Fog 

NIIRS  7.0 

Group  5 

1 

99 

99 

29.64 

5.04 

Night 

Fog 

NIIRS  7.0 

Group  5 

2 

100 

100 

31.43 

4.53 

Night 

Fog 

NIIRS  7.0 

Group  5 

3 

100 

100 

29.99 

3.29 

Night 

Fog 

NIIRS  7.0 

Group  4 

1 

55 

35 

30.89 

9.51 

Night 

Fog 

NIIRS  7.0 

Group  4 

2 

85 

80 

32.42 

7.66 

Night 

Fog 

NIIRS  7.0 

Group  4 

3 

91 

90 

0.03 

6.56 

Night 

Fog 

NIIRS  7.0 

Group  3 

1 

32 

16 

25.19 

6.16 

Night 

Fog 

NIIRS  7.0 

Group  3 

2 

59 

56 

32.01 

8.21 

Night 

Fog 

NIIRS  7.0 

Group  3 

3 

68 

66 

24.25 

7.10 

Night 

Fog 

NIIRS  8.0 

MQ-9 

1 

93 

92 

33.55 

7.40 

Night 

Fog 

NIIRS  8.0 

MQ-9 

2 

98 

98 

25.58 

5.80 

Night 

Fog 

NIIRS  8.0 

MQ-9 

3 

99 

99 

36.90 

5.30 

Night 

Fog 

NIIRS  8.0 

Group  5 

1 

97 

97 

22.47 

6.15 

Night 

Fog 

NIIRS  8.0 

Group  5 

2 

100 

99 

25.96 

4.70 

Night 

Fog 

NIIRS  8.0 

Group  5 

3 

100 

99 

— 

3.96 

Night 

Fog 

NIIRS  8.0 

Group  4 

1 

57 

35 

25.33 

10.09 

Night 

Fog 

NIIRS  8.0 

Group  4 

2 

85 

81 

40.99 

8.44 

Night 

Fog 

NIIRS  8.0 

Group  4 

3 

94 

93 

59.47 

6.98 

Night 

Fog 

NIIRS  8.0 

Group  3 

1 

29 

17 

22.68 

8.16 

Night 

Fog 

NIIRS  8.0 

Group  3 

2 

71 

65 

47.04 

9.03 

Night 

Fog 

NIIRS  8.0 

Group  3 

3 

82 

81 

58.47 

8.18 
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